|
Chris
Dove and Linda Phipps report on a method of evaluation community
networking projects again seven 'soft' criteria:
- Creativity
- Connectivity/connectedness
- Confidence
- Competencies
- Capacity
building — self-reliance and ‘ownership’
- Choice
- Content
Following
the Community Information Networks ’99 conference in
Sheffield, UK, a piece of ‘desk’ research has been
carried out by Chris Dove, formerly of Brixton Online and
Linda Phipps of St William’s Foundation.
This
is based on evaluating 7 pilot projects against a new set
of criteria, which we have called the 7 ‘C’s. The
aim is to develop a new model of evaluation, by creating a
new framework of evaluation for demonstration projects.
In
particular, we want to stimulate ideas on ‘soft’/qualitative
measures and outcomes, such as the percentage of participants
feeling more confident or interested in participation.
Please
contribute your comments and ideas by reading the attached
report of the desk research and posting any comment to the
CONET mailing list. To join conet
- mailto:majordomo@ukco.co.uk
the message, no subject
subscribe conet
You
will find more about the conet list on the Communities Online
Web site http://www.communities.org.uk
in the resources section.
You
can also contact Chris and Linda directly on c.dove@can-online.org.uk
and LLPhipps@aol.com.
CIN
‘99 Follow-up Proposal — draft report
Contents
- Aims
- Proposal
- Methodology
- Pilot
7 Cs questions
- Conclusions
and next steps
- The
pilot interview reports
1.
Aims
CIN
’99 was the conference of Community Information Networks.
It took place on 31 August — 1 September 1999, at the
University of Sheffield. It was a 2-day interactive conference,
aiming to bring together those already involved in or interested
in Community Information Networks. It provided up to date
views from the UK and Europe of current work by CINs, and
opportunities to visit projects, network, and share information.
One of the workshops generated an intense discussion on the
issue of evaluating the impact of ICTs in promoting social
inclusion, and reducing social exclusion. Participants felt
strongly that we needed to develop a new model of evaluation,
by creating a new framework of evaluation for demonstration
projects. In particular, we would aim to stimulate ideas on
‘soft’/qualitative measures and outcomes, such as
the percentage of participants feeling more confident or interested
in participation.
2.
Proposal
It
was proposed and agreed at CIN ’99 that a desk research
exercise be carried out by Chris Dove of Brixton Online and
Linda Phipps of St William’s Foundation, based on a new
set of criteria, which we called the 7 ‘C’s (pun
intended!):
- Creativity
- Connectivity/connectedness
- Confidence
- Competencies
- Capacity
building — self-reliance and ‘ownership’
- Choice
- Content
3.
Methodology
In
November 1999, 7 pilot projects at different stages of development
provided detailed commentaries questions contained in the
7 Cs questionnaire. In asking the questions, the aim was to
elicit information, to test the methodology and relevance
of the questions, and to create a basis for proposing a fuller
and more wide-ranging research proposal for probing
evaluative measures and frameworks and for exchange of good
practice, for which a bid for funding could be made.
The
pilot questions are set out below, followed by some general
conclusions, and then finally the information elicited from
each of the seven groups. Chris and Linda tested two slightly
different approaches, and we conclude that the most effective
method is to request written on-line completion of the 7 Cs
questionnaire, with clarification if required, followed by
a telephone interview. A report was prepared for each pilot
project, covering their responses to a set of questions on
each of the 7 criteria.
4.
Pilot questions
Creativity:
Defining
creativity as innovation, rather than a cultural or artistic
attribute, how does your project demonstrate creativity? Recognising
that creativity could be demonstrated in at least two phases
or levels — the design of the project, and its implementation,
how could creativity in the project be evaluated?
Connectivity/connectedness:
How
does your project demonstrate increased connectivity/connectedness
among your client communities? How could this be evaluated?
Confidence:
How
does your project demonstrate increased confidence among
your client communities? How could this be evaluated?
Competencies:
How
does your project increase community competencies? How could
this be evaluated?
Capacity
building — self-reliance and ‘ownership’:
How
does your project demonstrate community capacity building
— defining this as self-reliance and ‘ownership’?
How could this be evaluated?
Choice:
How
does your project increase and enable community choices? How
could this be evaluated?
Content:
How
does your project increase the relevant content available
to community groups and enable communication to be expanded
and enhanced? How could the contribution of content be evaluated?
Linda
Phipps Chris Dove
LLPhipps@aol.com c.dove@can-online.org.uk
5.
Conclusions
For
many projects, evaluation is not inherent in their work plan.
Much evaluation that is carried out is based on the need to
prove the delivery of the outputs required by funders —
largely quantitative, such as number of jobs created, number
of people trained to Level X. These kinds of measures are
not linked to qualitative outcomes, nor driven by the core
goals of the group running the project, nor by the needs or
wishes of their clients and beneficiaries. Few of the projects
we spoke to were commissioning from independent third parties,
or carrying out themselves, evaluation against measures based
on their own or clients goals and preferences.
Whilst
some demonstration measures are being used in projects, many
more potential ways in which the 7 Cs could be measured and
evaluated emerged in discussion. However, whilst such debates
proved very interesting, they did not necessarily trigger
a plan by the project being interviewed to start new areas
of evaluation. Clearly, whilst many groups would be interested
in undertaking evaluation, pressures of time and other scarce
resources currently preclude this. In today’s harsh funding
and delivery climates, if it is not necessary to survival
today, it will not get done. Hence part of our objective through
the proposed research project is to ensure that any new measures
we develop are perceived as relevant to the projects themselves.
1.
Creativity:
Creativity
may be deployed in the design as well as in the implementation
phases of a project. One project, targeted on older people
unfamiliar with the use of new technologies, aimed deliberately
to create a simple-to-use standardised template which does
not need customising or ingenuity. The real creativity of
this project has been in the behind the scenes complexity,
in enabling computer technology to be useful and accessible
to this relatively excluded group. It might also be argued
that the simple to use but cleverly constructed system enables
the latent creativity of older people to be released, in that
they can tell and record and share the story of their past
in a new and interactive way, expressing their emotional and
inner selves. The creativity in the design of the template
can be perceived in the quality of design, and the structure
of the product for content based image retrieval, and these
aspects could form the basis of an evaluation.
Another
project, based on networking of groups involved in health
projects, again builds its creativity essentially into project
design. It is based on using the inherent creativity of net/email
as a medium. In particular, it enables us to ask questions
to which we do not know the answer: we can post a question
by email, asking if anyone out there has any ideas, which
will ripple out to unknown individuals and groups — and
someone, or several groups, possibly strangers to us at present,
will answer. This requires us to have trust in the medium
that we can make contacts, and exchange ideas and questions
as well as facts and information. Individually, we can choose
whether to get involved or not, and in our own space and time.
This is a new way of working and will be able to demonstrate
the potential of such new ways of working — to find out
what that potential might be.
Use
of electronic communications is linked to the concept of creativity
as spontaneity. The electronic medium comes close to mimicking
the natural social interaction of conversation, with its non-sequiteurs,
sparking of ideas, unstructured "chaos", "nourishment"
and "warmth" of human interaction. Email interactions
span an enormous range and are very flexible: they can be
both very specific and focussed, and very fluid and dynamic.
Evaluation
could be structured around asking people if their participation
had been useful. Either retrospectively, or by creating a
baseline and comparing with later samples, we could examine
people’s attitudes to/fears of ICT. We might usefully
sample the SAME people later, for their views on usefulness,
enjoyment etc.
It
is crucial to see creativity as a skill which can be taught
and leant, rather than an innate attribute possessed by the
few. Creativity is about innovative approaches to problem-solving
— a capacity to conceptualise and embody and demonstrate
a new approach, or a new metaphor for a problem. It is about
developing new ways of thinking and developing thinking communities.
Artists
may be a vehicle to help enable groups to make just such a
conceptual leap. In one project focussed on young people,
the aim has been to put the use of accessible software into
frameworks of thinking and of community perceptions, as a
means of shifting individual thinking and their perception
of themselves and of their communities. Part of this activity
involves mapping communities. Evaluation could include development
of "soft" measures around changes in perceptions,
tracking the development of learning, and the evidence required
to achieve accreditation, including portfolios, staff reports,
group work, tapes, and feedback evaluations.
Another
project exemplified design innovation by using Wide Area Networks
as the cheapest
and fastest way of creating a significant number of users
capable of using broadband
technologies.
Implementation innovation lay in using Infra Red and microwave
technologies to link the WANs. The London borough hosting
this project was a leader in creating an extranet by linking
local WANs. With a private sector partner, it carried out
one of the first borough-based audits of local ICT infrastructure.
It deployed IR and microwave technologies to create an extranet.
It has also created a company limited by guarantee, which
is to seek registration as a Regional Public Telecommunications
Operator and establish itself as an ISP. This is believed
to be the first such organisation created in the UK since
Kingston Communications, albeit the new organisation is small
at present. The project suggests that creativity
can be evaluated effectively by peer review.
Innovation
in implementation came from using positively the large number
of Council tower blocks. IR laser and microwave both need
line of sight to work, so a 100Mb IR laser was used to link
the Council's server farm to the top of a neighbouring tower
block; then a 10Mb microwave was sued to distribute the connectivity
to two business centres and a TEC. More will be linked in
this way in the near future. Also, it is a case of what is
innovation? Most things have been tried somewhere and the
private sector has probably been using IR and microwave for
a while. But this is probably the first laser/microwave network
based on top of a Council tower block.
A
web-based group has deployed creativity in encouraging groups
and organisations to rethink how they present themselves.
Much of the web design has been driven by a few 'experts',
although none had had much experience before. Evaluation of
creativity might be related to the design process and feedback.
A
network of women’s groups has viewed the internet as
a tool and sought to embed its use in the working practices,
aims and objectives of women's organisations. It enables organisations
and individuals within them to learn to use ICT, and to use
ICT to learn, raise visibility, communicate, gather information
and collaborate. It also creates a learning network. The flexibility
and serendipitous nature of electronic networking has lead
to unexpected outcomes which projects and members have worked
with and built on. Evaluation could:
- compare
start plans with mid project assessments
- use
formal evaluation report (exists)
- use
feedback at end of project evaluation event (exists)
Finally,
a national children’s charity operating a wide range
of local projects suggests that creativity at project level
(e.g. in relation to work with young people, families and
community organisations) could be tested out as by asking
about participants aspirations, and the extent to which they
have been able to realise these. It could also involve evaluation
by an academic institution or through self evaluation by participants,
and the production of a report or a multi-media project which
contained an element of reflection on the process and the
outcomes from the work. Generally, it is noted that, if potential
beneficiaries have been actively involved, not merely 'consulted'
in the planning, then the task of evaluation will be made
much easier, as they will have already stated their priorities
and can then be asked whether their experience of both the
participation/planning and implementation have been satisfactory.
2.
Connectivity/connectedness:
How
does your project demonstrate increased connectivity/ among
your client communities? How could this be evaluated?
One
project increases connectedness among older people by enabling
them to contribute, exchange and review reminiscences in a
social setting. Many such groups are called "reminiscence
groups". The sharing of information, listening and talking,
creates social contact between older people, and, as this
project also creates contact with schools, it is also cross-generational.
The project is linked to the regional, UK and European Reminiscence
Networks, which are developing measures of the benefits of
reminiscence. Benefits are considered to include improvements
to physical and mental health, with the sustainability of
alertness being a factor in delaying the moment of entry into
residential homes. Inclusivity is a feature in that new people
are gradually brought in to the groups. There would be a value
in being able to quantify such benefits.
In
setting up a community archive, a local group has to set itself
up, get funding, get premises/accommodation for the collection
and the equipment, and arrange copyright of the material.
Evidence from established groups is that the frequency of
group meetings grows over time. Local groups, established
up to 4 years ago, meet around 3 times a week. Evaluation
of impact on connectivity could be measured by frequency of
meetings, number of attendees, interactions between groups
(for example, group members were recently invited to visit
a newly formed group in Newry to give advice on how to go
about setting up a group). Partnerships are also formed with
local schools and residential homes for the elderly.
Another
measure of impact may be the existence at a Europe wide level
of a network of the organisers of the community archive groups,
and measures could be developed of how active it is and around
what issues.
Connectivity
and connectedness are the core objectives of projects which
are building national networks, rather than being specifically
geographically based. Connectivity could be evaluated through
a "species diversity" or ecological model —
plotting the growth of diversity of interactions over time.
Starting with some groups new to the use of the net/email,
we could investigate their growing use of this medium over
time. This could be measured in terms of numbers of messages
sent/received, content, kinds of messages and interactions.
One
project working with young people is actively considering
connectivity/connectedness. They are considering —
how does an area/community work — is a community purely
geographical/territorial? Is it based on networks? They are
looking at the definition of community and mapping this on
the basis on local perceptions, working with local organisations
and schools. The aim is to work with people’s own definitions
of themselves, their memories and experiences and associations
with place and community, with facilitation by a local artist,
and recognising that descriptions can be fictionalised as
well as factual. They will also be creating a baseline study,
so that changes over time, as people develop a sense of communities
of interest and on-line electronic communities as well as
of physical place, can be tracked.
One
project is enhancing connectedness among local organisations
by deploying mailing list software. Connectedness could be
evaluated on the basis of the range and number of organisations
that have staff participating on the email lists, the number
of eMails sent and the number of people sending them.
A
web based project has aimed at breaking down barriers and
offering local people services which meet their needs. The
web itself has been important in developing links with the
other local agencies. The Web project is also encouraging
them to use internet and email, and to develop awareness of
the potential for sharing, information, good practice and
common problems that a CIN can offer. This might be evaluated
by tracking the opinions and/or behaviour of these groups
— in terms of usage of electronic media — over time.
A
project to create a network of women’s groups suggests
that evaluation could look at the attendance at events, the
communication between events, outcomes from contacts, and
the stated benefits of contacts.
Finally,
a national children’s charity plans to calculate the
increase in usage of ICT among local voluntary and community
organisations with whom they are working, between the beginning
and end of the scheme. They point out that issues of sustainability
and the 'quality' of people's experience of ICT are as important
as the absolute numbers of people using/having access to ICT,
and could be evaluated by seeking their views.
3.
Confidence:
How
does your project demonstrate increased confidence among
your client communities? How could this be evaluated?
Dti
research has shown that, on average, older people have the
least confidence and least familiarity with use of new technology.
Additionally, they may become isolated with the loss of friends
and mobility, and fears of crime may result in self-imposed
isolation. Community archive groups are one way to break this
cycle, and, through the accessible use of relevant and interesting
content, incidentally provide people with access to computer
skills, and even public speaking skills, through talking to
other groups. These skills build confidence, and participants
(beneficiaries) could be asked about this aspect.
Self
confidence grows as people develop familiarity with and use
of the new technologies. This may not be a linear journey
— groups and individuals who achieve a level of confidence
can subsequently lose confidence after a setback. Measurements
could be developed around the %ages feeling unsure about doing
X, and then becoming at ease with technical applications.
One
project is creating a consortium including two community partners,
which has successfully bid for funding under ESF Priority
4. This has enabled the recruitment of a multimedia research
worker. Through computer "taster sessions", the
needs of young people are being established. In parallel,
the local University is carrying out an organisational audit
of each partner company, to establish their capacity to deliver
creative technologies. This audit covers organization skills
and level of interest in ICTs, their organizational structure,
budget, database, and training needs. The project is also
working with employers — through a business liaison officer
— to create "thumbnail sketches" of what new
employees employers will need. This enables the project to
ensure that the skills it is passing on to young people match
the demands of the local world of work.
In
terms of evaluation, it is recognised that this measurement
of current organizational capacity and of business skills
could be repeated in say 2 years’ time, creating both
a baseline and a way of tracking future changes in the media
sector. For example, in future there will be changes in the
local structure of employment and workers in this sector may
be employees, freelances, or set up their own companies, or
work in groups — all with implications for skills needed.
Evaluation may need to be at a variety of levels — individual
skills, teamwork capacities, the employment structure of the
sector, and also the broader level of the impact of creative
technologies on the wider community. Evaluation could be through
live projects, and make use of staff and peer group feedback
and appraisal.
One
project has suggested that a practical demonstration of increased
confidence arises from the completion of their own video by
a related Online Young People’s group, which is to be
screened on the web after its public launch in December '99.
A
Web based project is seeking to raise the confidence of both
information seekers and information providers, by providing
a web site that is easy to use and encourages further exploration,
by supporting users at public access points e.g. libraries
and community centres, and supporting those wishing to develop
their own web site through one to one 'as need' training.
This could be evaluated by seeking the opinions of information
seekers and information providers.
A
project to create a network of women’s groups starts
from an awareness of the constraints that the member organisations
work within, and the anxieties and concerns that they had
at the start of their participation in the networking project.
These can be compared with the feedback at a series of events
they have held. Further evaluation can be based on the steadily
increasing amount of usage amongst members, the amount of
mutual support they have given, the number of initiatives/innovations
in training they have developed for themselves, and the language
they now use.
A
national charity monitors the type of messages and calls received,
requesting technical or other support and assistance. It can
also evaluate its impact through the numbers attending Workshops.
4.
Competencies:
How
does your project increase community competencies? How could
this be evaluated?
Designs
of easy-to-use websites, encouraging further exploration,
skill developments including shared skill banks to cope with
current technologies, are key features of determinants of
competencies. A distinction is made between the building of
the competencies of organisations and those of the individuals
within them. Emphasis is placed on the increased ability of
different organisations to work in partnership to achieve
both on- and off-line goals: organisations are providing services,
campaigning or networking amongst their own members. The impact
on the organisation of increased skills is felt amongst individual
service users through enabling users to learn ICT skills,
through training courses etc. As skills are developed in this
way, increased demands can be made on managers and ICT support
staff which, given the lack of resources in most cases, often
leads to individuals researching their own needs, thus increasing
their knowledge.
These
impacts could be evaluated by the numbers of individuals progressing
from informal self-help sessions to enrolling on courses,
numbers tapping into tutor expertise as and when they need
it, with others benefiting from their experiences within their
courses. Comparing the starting points with the current position,
and assessing feedback and formal evaluations from individuals
and organisations, allows a measure of increased community
competencies. In one case this had led directly to an increase
in the number of people able to create their own contributions
to the local community newspaper, even e-mailing them in from
other locations. This has additionally led to some communities
undertaking their own fundraising activities and creating
their own promotional materials, leaflets, posters etc.
5.
Capacity building — self-reliance and ‘ownership’:
How
does your project demonstrate community capacity building
— defining this as self-reliance and ‘ownership’?
How could this be evaluated?
Youth
projects, ethnic and non-ethnic women’s projects, and
child and parent projects all demonstrated the degrees of
self-reliance and ‘ownership’ experienced by participants.
Groups feel in control of their communities and see IT as
a valuable resource, many meeting on a regular basis and playing
an active role in community events. Mutual support has developed
to form continuation groups where individuals finding common
ground can share specific IT skills - for example, associated
with searching the net. Women’s organisations in particular
offer self-help and personal development from within their
communities.
For
all, greater communication and transparency has been a key
benefit of the ICTs. Participants’ positive feedback
are themselves testimonies of the impact of ICTs on individuals
who are having their capacity ‘built’: they are
able to identify their own needs, to suggest possible means
of meeting those needs and the type of delivery that would
best suit them.
6.
Choice:
How
does your project increase and enable community choices? How
could this be evaluated?
The
creation and implementation of cross-sector strategies to
secure the best that information technology has to offer local
people, is a principle aim of the pilot initiatives. The development
of such strategies has involved a wide range of people from
a variety of organisations. Through such projects, participants
have been able to play a part in the design and planning of
local initiatives including, in one instance, residents’
involvement in the planning of a shopping and community centre
on their estate.
E-mail
and online discussion forums provide one measurement of enabled
community choices. Projects demonstrate the increased range
of opportunities for individual and local organisations in
terms of connectivity and online services and the opportunities
these provide for future on- and off-line communication. In
addition, organisations are strengthened by being run from
within the community. Again, self-evaluation amongst organisations
about their experiences using ICTs can be traced back to their
contributions via the net.
7.
Content:
How
does your project increase the relevant content available
to community groups and enable communication to be expanded
and enhanced? How could the contribution of content be evaluated?
The
overall aim of all projects is to promote the use of the Internet
within their communities. One project is doing this by making
relevant content more easily available by providing a single
search engine that will index all local websites. This gateway
site will also make it far easier to contact local groups
and resources. The provision of useful, accurate and up-to-date
content is desired. However, another project sees its site
as being as very much a portal into the net as a local information
service.
Others
aim to enable organisations to create their own websites and
discuss best web practices, seeing the presence of sites on
the net as itself creating a resource for individuals and
organisations to gather and exchange information.
E-mail
lists have been the main enhancement of communication in the
speedy and effective distribution of online content. Given
that many projects exist as part of wider local initiatives,
it is difficult to evaluate and separate the benefits that
have come from the contribution of content. Whilst statistics
from site hits and visits are obvious measurements of the
pace of communication, they do not indicate the increased
expansion and enhancement of relevant content to individuals
or groups as this is would require more of a subjective measurement.
What
do we want to happen next?
First
of all, we would like all of you CONetters to add your
views and comments so that we can improve and develop
this report and ideas, ideally by 24 January and definitely
by the end of January please.
Second,
we would like to make a submission to government — the
DTI, possibly Europe — to fund an action-based research
project to develop the work so far. We think that this funding
could be on two levels — for individual projects, and
for overarching development, evaluation and comparison of
measures. In particular, we think that we could create some
core measures for evaluation, which a small group of projects
could sign up to use in the first instance. Evaluation takes
time and effort, so that within the research project, they
should receive funding to introduce these measures. (They
could also develop specific measures of their own.) Using
the core measures, we could create a current baseline or benchmark,
so that we could make comparisons across projects. By repeating
the evaluation at intervals, we could also track changes within
projects, over time. We would also propose to build in contact
with the projects’ beneficiaries, so that evaluation
from their viewpoint could be included.
The
aim would be to develop a process of real learning from projects
at different stages of development, and practical and effective
ways of sharing this good practice for the benefit of all
projects. Our concept is also to be as inclusive as possible,
so as many projects as possible should be encouraged to participate
in developing and applying the core measures, and reflecting
on their use.
6.
The pilot interview reports
The
following projects kindly agreed to be interviewed and their
individual reports follow:
COMMA
— chris.levack@geo2.poptel.org.uk
Healthy
Living Project — yu07@dial.pipex.com
Switched
On — pavion-info@dial.pipex.com
and info@pavilion.org.uk
The
Madeley Web — sos@stiperstones.co.uk
Newham
Online — richard.stubbs@newham.org.uk
Women
Connect — co-ord@womenconnect.org.uk
The
Children’s Society — ajd@childsoc.org.uk
7
Cs Interview with COMMA, Batley
Methodology
7
‘C’s pilot questions and criteria sent in advance
of telephone interview.
About
COMMA
Community
Multimedia Archives are electronic databases of photographs,
text and oral history recordings produced by local community
groups using specially designed COMMA software, a PC, scanner
and sound recording equipment. The resulting community archive
is usually cut onto a CD and distributed locally to schools,
libraries etc. Recently, a website
www.commanet.org
has been set up and fully searchable local archives are beginning
to be put up on the site. The project started in 1994 as a
City Challenge initiative in Batley, West Yorkshire and has
now spread to become a European wide initiative.
Contact
is: Chris Levack, Bretton Hall, West Bretton, Wakefield, WF4
4LG
Tel.
01924 832114, email chris.levack@geo2.poptel.org.uk
1.
Creativity:
Defining
creativity as innovation, rather than a cultural or artistic
attribute, how does your project demonstrate creativity? Recognising
that creativity could be demonstrated in at least two phases
or levels — the design of the project, and its implementation,
how could creativity in the project be evaluated?
The
COMMA project is a good example of the deployment of creativity
in the design rather than implementation phases. Indeed, as
the target market is older people unfamiliar with the use
of new technologies, the approach has been to create a simple-to-use
standardised template which does not need customising or ingenuity.
The real creativity of this project has been in the behind
the scenes complexity, in enabling computer technology to
be useful and accessible to this relatively excluded group.
It might also be argued that the simple to use but cleverly
constructed system enables the latent creativity of older
people to be released, in that they can tell and record and
share their story in a new and interactive way, expressing
their emotional and inner selves. The creativity in the design
of the template can be perceived in the quality of design,
and the structure of the product for content based image retrieval.
2.
Connectivity/connectedness:
How
does your project demonstrate increased connectivity/connectedness
among your client communities? How could this be evaluated?
Contributing
and reviewing reminiscences is done in a social setting, stimulating
social contact — listening and talking - between older
people, and also cross-generational interactions. The sharing
of information triggers the contribution and exchange of reminiscences,
and many such groups are called "reminiscence groups".
The project is linked to the regional (Yorkshire & Humber)
Reminiscence Network, and to the UK and European Reminiscence
Networks, which are developing measures of the benefits of
reminiscence. Benefits are considered to include improvements
to physical and mental health, with the sustainability of
alertness being a factor in delaying the moment of entry into
residential homes. Inclusivity is a feature in that new people
are gradually brought in to the groups. There would be a value
in being able to quantify such benefits.
In
setting up an archive, a local group has to set itself up,
get funding, get premises/accommodation for the collection
and the equipment, and arrange copyright of the material.
Evidence from established groups is that the frequency of
group meetings grows over time. The Batley group was established
over4 years ago, and still meets regularly. The more recently
established Grimethorpe group now meets 3 times a week. Evaluation
of impact on connectivity could be measured by frequency of
meetings, number of attendees, interactions between groups
(for example, group members were recently invited to visit
a newly formed group in Newry to give advice on how to go
about setting up a group). Partnerships are also formed with
local schools and residential homes for the elderly.
Another
measure of impact may be the existence at a Europe wide level
of a network of the organisers of the community archive groups,
and measures could be developed of how active it is and around
what issues.
3.
Confidence:
How
does your project demonstrate increased confidence among
your client communities? How could this be evaluated?
Dti
research has shown that, on average, older people have the
least confidence and least familiarity with use of new technology.
Additionally, they may become isolated with the loss of friends
and mobility, and fears of crime may result in self-imposed
isolation. Community archive groups are one way to break this
cycle, and, through the accessible use of relevant and interesting
content, incidentally provide people with access to computer
skills, and even public speaking skills, through talking to
other groups. These skills build confidence, although at the
moment COMMA have no plans to ask people about this aspect.
4.
Competencies:
How
does your project increase community competencies? How could
this be evaluated?
Community
archive groups learn how to use multimedia, including word
processing, scanning, editing sound, video editing. They must
also develop competencies in fund-raising, the construction
of information, accounting, self-organisation and the capacity
to develop partnerships.
5.
Capacity building — self-reliance and ‘ownership’:
How
does your project demonstrate community capacity building
— defining this as self-reliance and ‘ownership’?
How could this be evaluated?
The
project empowers older people to create and own their own
records and on-line archive — to document their community’s
cultural heritage. Working as a group builds capacity for
the group to take joint responsibility.
6.
Choice:
How
does your project increase and enable community choices? How
could this be evaluated?
The
groups make choices around what material goes onto their archive,
and their web-site if any. They make joint decisions around
how the material may be used by third parties, and the management
of copyright.
7.
Content:
How
does your project increase the relevant content available
to community groups and enable communication to be expanded
and enhanced? How could the contribution of content be evaluated?
The
project is all about content — interesting, local content
of relevance to older people and to their communities.
7
Cs Interview with the Healthy Living Project
Methodology
7
‘C’s pilot questions and criteria sent in advance
of telephone interview.
About
the Healthy Living Project
The
Healthy Living project is funded by the Dept of Health for
3 years. It is currently in the development phase. Its aim
is to use ICT to network projects involved in health, and
to create communities of interest. Its role is not to signpost
groups to where there is officially provided information on
diagnosis. Rather, it is to help groups to find out about
"how to" through contacting and learning from other
groups. For example, groups may wish to find out about how
to involve the local community in consultation. The project
aims to act as a catalyst for the Department of Health, supporting
it in implementing the WP "Our Healthier Nation",
though the medium of web-sites and the internet. In particular,
it will enable interactions and partnerships of several groups
simultaneously through email groups, rather than the 1 to
1 interaction of meetings and telephone.
Contact
is: David Lloyd (tel/fax 01588 630344, email yu07@dial.pipex.com)
1.
Creativity:
Defining
creativity as innovation, rather than a cultural or artistic
attribute, how does your project demonstrate creativity? Recognising
that creativity could be demonstrated in at least two phases
or levels — the design of the project, and its implementation,
how could creativity in the project be evaluated?
The
creativity of this project lies in its design. It is based
on using the inherent creativity of net/email as a medium.
In particular, it enables us to ask questions to which we
do not know the answer: we can post a question by email, asking
if anyone out there has any ideas, which will ripple out to
unknown individuals and groups — and someone, or several
groups, possibly strangers to us at present, will answer.
This requires us to have trust in the medium that we can make
contacts, and exchange ideas and questions as well as facts
and information. Individually, we can choose whether to get
involved or not, and in our own space and time. This is a
new way of working and the benefit for the Dept of Health
from this particular project is to be able to demonstrate
the potential of such new ways of working — to find out
what that potential might be.
This
way of working is linked to the concept of spontaneity. This
medium comes close to mimicking the natural social interaction
of conversation, with its non-sequiteurs, sparking of ideas,
unstructured "chaos", "nourishment" and
"warmth" of human interaction. David noted that
email interactions were very flexible, and could be both very
specific and focussed, and very fluid and dynamic.
Evaluation
could be structured around asking people if their participation
had been useful. Either retrospectively, or by creating a
baseline and comparing with later samples, we could examine
people’s attitudes to/fears of ICT. We might usefully
sample the SAME people later, for their views on usefulness,
enjoyment etc.
2.
Connectivity/connectedness:
How
does your project demonstrate increased connectivity/connectedness
among your client communities? How could this be evaluated?
Connectivity
and connectedness are the core objectives of the project,
which is based on national networking rather than being specifically
geographically based. In relation to health topics, including
longer life, it aims to enable groups of people to feel part
of a wider community with aims such as reducing accidents
among older people.
This
could be evaluated through a "species diversity"
or ecological model — plotting the growth of diversity
over time. Starting with some groups new to use of the net/email,
it could investigate their growing use of this medium over
time. This could be measured in terms of numbers of messages
sent/received, content, kinds of messages and interactions.
3.
Confidence:
How
does your project demonstrate increased confidence among
your client communities? How could this be evaluated?
Self
confidence grows as people develop familiarity with and use
of the new technologies. David notes that this is not a linear
journey — groups and individuals who achieve a level
of confidence can subsequently lose confidence after a setback.
He sees the development of confidence as being rather like
tasting new foods! Measurements could be developed around
the %ages feeling unsure about doing X, and then becoming
at ease with technical applications.
4.
Competencies:
How
does your project increase community competencies? How could
this be evaluated?
Competencies
developed obviously include using ICT for word processing
etc. However, additionally, competencies are developed in
crating partnerships and networking, and in capacity to use
electronic information. These might be measured by examining
the actual use made of the net and email, by numbers of groups
contacted, and by capacity to use equipment — fax, email
etc. It is important to recognise the interaction of technical
and social competencies.
5.
Capacity building — self-reliance and ‘ownership’:
How
does your project demonstrate community capacity building
— defining this as self-reliance and ‘ownership’?
How could this be evaluated?
The
aspiration of the Dept of Health is that, by the end of the
project, th interest of the participating groups will enable
the networks to be sustained.
6.
Choice:
How
does your project increase and enable community choices? How
could this be evaluated?
Choice
can be exercised on different levels. The project is targeted
on electronic interactions, so does not give choice of medium
to use. However, it aims to use this medium as a vehicle to
enable groups to have the choice of a much wider and potentially
more inclusive circle of contacts. This could be measured
by their take up of ‘mass’ contact email group communications
as opposed to one to one contacts such as personal letters
or telephone calls.
7.
Content:
How
does your project increase the relevant content available
to community groups and enable communication to be expanded
and enhanced? How could the contribution of content be evaluated?
The
project aims to build access to content in terms of what groups
want to know. For example, it may cover "frequently asked
questions, evaluation, or signposting to clusters of web-sites
on subjects such as good practice in generating community
participation, or in creating partnerships for a safer neighbourhood.
Content
could be evaluated by asking users about usefulness, and by
tracking what they do with it — numbers of groups they
contact or forge stronger links with or visit.
7
‘C’s Interview with Switched On, Leeds
Methodology
7
‘C’s pilot questions and criteria sent in advance
of telephone interview.
About
Switched On
The
Switched On project, based in Leeds, is aimed at disadvantaged
young people and uses IT to enable them to develop skills
and meet their own needs.
Contact
is: Sue Ball, Pavilion, 2 Woodhouse Square, Leeds LS3 1AD,
tel 0113 243 1749, fax 245 3329, email pavilion-info@dial.pipex.com
and info@pavilion.org.uk
1.
Creativity:
Defining
creativity as innovation, rather than a cultural or artistic
attribute, how does your project demonstrate creativity? Recognising
that creativity could be demonstrated in at least two phases
or levels — the design of the project, and its implementation,
how could creativity in the project be evaluated?
It
is crucial to see creativity as a skill which can be taught
and leant, rather than an innate attribute possessed by the
few. Creativity is about innovative approaches to problem-solving
— a capacity to conceptualise and embody and demonstrate
a new approach, or a new metaphor for a problem. It is about
developing new ways of thinking and developing thinking communities.
In
running a workshop for young people, Switched on has worked
with a team of artists to help enable a group of young people
to make just such a conceptual leap. The aim has been to put
the use of accessible software into frameworks of thinking
and of community perceptions, as a means of shifting individual
thinking and their perception of themselves and of their communities.
Part of this activity involves mapping communities, and training
accredited by the Open College will be offered.
Evaluation
is a new concept here, however possibilities include development
of "soft" measures around changes in perceptions,
tracking the development of learning, and the evidence required
to achieve accreditation, including portfolios, staff reports,
group work, tapes, and feedback evaluations.
2.
Connectivity/connectedness:
How
does your project demonstrate increased connectivity/connectedness
among your client communities? How could this be evaluated?
This
is an area of active development by Switched On. They are
considering — how does an area/community such as Chapeltown
work — is a community purely geographical/territorial?
Is it based on networks? They are looking at the definition
of community and mapping this on the basis on local perceptions,
working with local organisations and schools. The aim is to
work with people’s own definitions of themselves, their
memories and experiences and associations with place and community,
with facilitation by a local artist, and recognising that
descriptions can be fictionalised as well as factual. Switched
On will create a baseline study, so that changes over time,
as people develop a sense of communities of interest and on-line
electronic communities, as well as of physical place, can
be tracked.
3.
Confidence:
How
does your project demonstrate increased confidence among
your client communities? How could this be evaluated?
Switched
On is creating a consortium including 2 community partners,
which has successfully bid for funding under ESF Priority
4. This has enabled the recruitment of a multimedia research
worker. Through computer "taster sessions", the
needs of young people are being established. In parallel,
Leeds University is carrying out an organisational audit of
each partner company, to establish their capacity to deliver
creative technologies. This audit covers organization skills
and level of interest in ICTs, their organizational structure,
budget, database, and training needs. The project is also
working with employers — through a business liaison officer
— to create "thumbnail sketches" of what new
employees employers will need. This enables the project to
work to ensure that skills passed on to young people are relevant
to the local world of work and its demands, and the plan is
to revisit the Business Plan to build in this flexible response,
and also to seek to deliver training etc through communities
and through their own resources.
In
terms of evaluation, it is recognised that this measurement
of current organizational capacity and of business skills
could be repeated in say 2 years’ time, creating both
a baseline and a way of tracking future changes in the media
sector. For example, in future there will be changes in the
local structure of employment and workers in this sector may
be employees, freelances, or set up their own companies, or
work in groups — all with implications for skills needed.
Evaluation may need to be at a variety of levels — individual
skills, teamwork capacities, the employment structure of the
sector, and also the broader level of the impact of creative
technologies on the wider community. Evaluation could be through
live projects, and make use of staff and peer group feedback
and appraisal.
4.
Competencies:
How
does your project increase community competencies? How could
this be evaluated?
Switched
On has focussed very much on enabling young people to develop
proficiency in the use of software, as a means to do more
than simply download from the web the images of others. Their
approach has been very much about enabling young people to
use software to create and develop their own cultural interests
and images.
Switched
On is working with Leeds University to define relevant competencies,
both in terms of technical skills and in terms of ability
to communicate. Professor Roger Hartley is examining the potential
of "empowerment evaluation", developed in America
and building this aspect into project methodology. It is based
on how projects work with communities and enable them to identify
their own indicators of performance.
5.
Capacity building — self-reliance and ‘ownership’:
How
does your project demonstrate community capacity building
— defining this as self-reliance and ‘ownership’?
How could this be evaluated?
The
project focus is on community organizations and growing their
capacity to deliver. This could be measured by the extent
to which they capture web information and disseminate information.
The organizational audit will create a benchmark of current
capacity, and if rerun in say 2 years, will track change.
6.
Choice:
How
does your project increase and enable community choices? How
could this be evaluated?
Switched
on feel that this is an interesting and difficult area. Excluded
communities by definition lack choices. And yet when we seek
to expand choice, it remains within circumscribed boundaries.
On line communities created by young people around their own
priorities may wish to publish information on drug dealing.
So this concept raises issues and empowerment, governance,
and giving young people responsibility and options to behave
as a community.
7.
Content:
How
does your project increase the relevant content available
to community groups and enable communication to be expanded
and enhanced? How could the contribution of content be evaluated?
Switched
on has always seen the use and development of relevant content
as the key to involving excluded young people and introducing
them to access to web-based content. This could be measured
by the number of hits on their own (or other) sites, the volume
of links to other sites, and "production values"
(the quality of appearance of the site). At the moment they
have no plans to research this issue.
7
‘C’s Interview with Maddeley Web
Methodology
7
‘C’s pilot questions and criteria sent online for
online completion, and supplemented by telephone interview.
About
Maddeley Web
The
Madeley Web project provides a host seb site and is aimed
at encouraging local groups and agencies to use the internet
and email in a Challenge Fund area.
Contact
is: Geoff Sproson, email sos@stiperstones.co.uk
1.
Creativity:
The
web has involved a lot of creativity in encouraging groups
and organisations to rethink how they present themselves.
Much of the web design has been driven by a few 'experts'
although none had had much experience before. It is difficult
to evaluate the creativity that went on but the project is
learning more about design and the design process and from
feedback.
2.
Connectivity/connectedness:
There
was already a lot of inter agency activity in the area, the
whole Challenge Project has been aimed at breaking down barriers
and offering local people services which meet their needs.
A recent report from our Neighbourhood Action Team, attached,
an idea of the
types
of organisations involved.
The
web itself has been important as the group meets with all
the other local agencies. The Maddeley Web project is encouraging
them to use internet and email. Many of the agencies have
yet to understand the potential for sharing, information,
good practice and common problems that a CIN can offer. There
is a long way to go before internet and email will replace
or effectively build on existing connections.
3.
Confidence
The
Maddeley Web seeks to raise the confidence of both information
seekers and information providers by providing a web site
that is easy to use and encourages further exploration, by
supporting users at public access points e.g. libraries and
community centres, and supporting those wishing to develop
their own web site through one to one 'as need' training.
More information is available on the Parish Council and Madeley
Writers sections of the web site.
4.
Competencies
The
web encourages individuals to develop skills and encourages
groups towards developing a shared skill bank to cope with
current technologies. Some individuals have progressed from
informal help sessions to enrolling on courses, others tap
into the project team’s expertise as and when they need
it. Often these sessions lead to a sharing of knowledge and
the project team’s own competencies grow as well.
5.
Capacity building - self-reliance and 'ownership'
Two
groups in particular have developed these concepts through
a range of experiences of which the web has been one. The
Asian Womens group now meet regularly, play an active group
role in community events, attend college courses and share
experiences with other using email. The Before Parenthood
and Beyond Group have formed a continuation group of their
own and are developing IT skills associated with bringing
up young children, searching the internet, producing greetings
cards and printing t-shirts.
6.
Choice
The
web was involved in a Planning for Real project which allowed
residents of Sutton Hill to play a part in the redesign of
the shopping and community centre of their estate. The number
of opinions expressed online was small but the opportunity
for future uses was clear.
7.
Content
The
main aim of the project is to promote use of the internet
within the community. The number of users at the moment does
not make it a viable way of providing local information. The
project aims to keep the content useful, accurate and up to
date but sees the web as being as much a portal into the internet
as a local information service.
Evaluation
The
Madeley Web was created and exists as a part of the much wider
Madeley Challenge Initiative. It is impossible to separate
the benefits that CIN has brought from the benefits generated
by all the other initiatives. Evaluations expressed in quantifiable
terms have many limitations, and the time that can be spent
in trying to collect such information is something that this
project can not afford.
The
whole project is being evaluated by an outside organisation
in the near future, and a report will be made available when
it is published.
7
‘C’s Interview with Newham Online
Methodology
7
‘C’s pilot questions and criteria sent online for
online completion, and supplemented by telephone interview.
About
Newham Online (NeOn)
The
London Borough of Newham is aiming to be ‘the most wired
in the UK’, and created an extranet by linking local
WANs. Newham Online (NeOn) was one of first to deploy IR and
microwave technologies to create an extranet. NeOn and its
partner Marconi carried out one of the first borough-based
audits of local ICT infrastructure. NeOn has just created
newham.net limited as a company limited by guarantee which
is to seek registration as a Regional Public Telecommunications
Operator and establish itself as an ISP.
Contact
is: Richard Stubbs, email R_Stubbs@compuserve.com
1.
Creativity:
Defining
creativity as innovation, rather than a cultural or artistic
attribute, how does your project demonstrate creativity? Recognising
that creativity could be demonstrated in at least two phases
or levels — the design of the project, and its implementation,
how could creativity in the project be evaluated?
Design
innovation lay in observing that linking existing Wide Area
Networks was the
cheapest
and fastest way of creating a significant number of users
capable of using broadband
technologies.
Implementation innovation lay in using Infra Red and microwave
technologies to link the WANs. Creativity is best evaluated
by peer review, and Newham is one of
first
boroughs in which an extranet has been created by linking
local WANs. Newham Online (NeOn) with its partner Marconi
carried out one of the first borough-based audits of local
ICT infrastructure. NeOn was one of first to deploy IR and
microwave technologies to create an extranet. NeOn has just
created newham.net limited as a company limited by guarantee
which is to seek registration as a Regional Public Telecommunications
Operator and establish itself as an ISP. This is believed
to be the first such organisation created in the UK since
Kingston Communications, albeit the new organisation is small
at present.
What's
innovative about it? Innovation in implementation came from
observing that Newham had a lot of Council tower blocks and
that if you stood on the top of one of them you could see
a great deal (obvious really). IR laser and microwave both
need line of sight to work, so a 100Mb IR laser was used to
link the Council's server farm to the top of a neighbouring
tower block; then a 10Mb microwave was sued to distribute
the connectivity to two business centres and a TEC. More will
be linked in this way in the near future. Also, it’s
a case of what is innovation? Most things have been tried
somewhere and the private sector has probably been using IR
and microwave for a while. But this is probably the first
laser/microwave network based on top of a Council tower block.
2.
Connectivity/connectedness:
How
does your project demonstrate increased connectivity/connectedness
among your client communities? How could this be evaluated?
The
project’s client communities are mainly the people that
work in academic, public, private and voluntary sector organisations
in Newham. The main means of enhancing connectedness has been
the deployment of mailing list software. The only lists aimed
at the community in general have been established by Newham
Young People Online which is a self managing community that
NeOn members have helped to develop. Connectedness could be
evaluated on the basis of the range and number of organisations
that have staff participating on the email lists, the number
of eMails sent and the number of people sending them.
3.
Confidence:
How
does your project demonstrate increased confidence among
your client communities? How could this be evaluated?
The
best example of increased confidence within the NeOn umbrella
of projects is probably
to
be found amongst NYPO members, who recently completed their
own video about NYPO. This is to be screened on the web after
its public launch on 9th December '99.
4.
Competencies:
How
does your project increase community competencies? How could
this be evaluated?
The
main community competency which has been increased is the
ability of different
organisations
to work in partnership to achieve on and offline goals. This
could be evaluated by asking the organisations concerned.
5.
Capacity building — self-reliance and ‘ownership’:
How
does your project demonstrate community capacity building
— defining this as self-reliance and ‘ownership’?
How could this be evaluated?
NYPO
is an online learning community of young people that has been
nurtured by NeOn. The member's of this group have a considerable
feeling of ownership and control of their community and see
it as a valuable resource. This has been evaluated by independent
academic research.
6.
Choice:
How
does your project increase and enable community choices? How
could this be evaluated?
NeOn
exists to create and implement a cross sector strategy that
aims to secure the best that
the
information society has to offer for local people. The development
of the strategy has
involved
a wide range of people from a wide range of organisations,
although not individual members of the community. NeOn has
increased the range of opportunities for local organisations
in terms of connectivity and online services. This could be
evaluated by independent research asking NeOn participants
their opinions.
7.
Content:
How
does your project increase the relevant content available
to community groups and enable communication to be expanded
and enhanced? How could the contribution of content be evaluated?
Relevant
content will be made more easily available, by providing a
single search engine that
will
index all local web sites. This gateway site will also make
it far easier to contact local
groups
and resources. The main enhancement of communication has been
through the
use of eMail lists.
7
‘C’s Interview with Women Connect
Methodology
7
‘C’s pilot questions and criteria sent online for
online completion, and supplemented by telephone interview.
About
Women Connect
Women
connect has taken the internet as a tool and sought to embed
its use in the working practices, aims and objectives of women's
organisations. The internet is a means and an ends - we are
enabling organisations and individuals within them to learn
to use ICT and to use ICT to learn, raise visibility, communicate,
gather information and collaborate. The project
aims
to build organisations’ ability to use ICT, and to enable
them to use ICT's to build ‘capacity’. In the same
way we are creating a network to learn and also learning to
network. These aims are both inputs and outcomes!
With
the assistance of project consultants/trainers, the groups
who came into membership have developed ideas about how to
use ICTs and how to develop a whole organisation approach.
The starting framework has been just that — a starting
point - not a straight jacket, and the flexibility and serendipitous
nature of electronic networking has lead to unexpected outcomes
which projects and members have worked with and built on.
Contacts
are: Marion Scott, email m.scott@iname.com, and
MargaretPage@maya-consultancy.demon.co.uk,
Joint Project Co-ordinators,
Women
Connect - using and shaping the Internet together
c/o
Community Development Foundation, 60 Highbury Grove, London
N5 2AG
e-mail
: co-ord@womenconnect.org.uk,
www.womenconnect.org.uk
tel
: 0171 226 5375 and 0171 354 2714, fax : 0171 704 0313
1.
Creativity:
Defining
creativity as innovation, rather than a cultural or artistic
attribute, how does your project demonstrate creativity? Recognising
that creativity could be demonstrated in at least two phases
or levels — the design of the project, and its implementation,
how could creativity in the project be evaluated?
*compare
start plans with mid project assessments
*use
formal evaluation report (exists)
*
use feedback at end of project evaluation event (exists)
2.
Connectivity/connectedness:
How
does your project demonstrate increased connectivity/connectedness
among your client communities? How could this be evaluated?
The
project was set up to create a network - this has been done
to an extent. We can look at the attendance at events, the
communication between events, outcomes from contacts, stated
benefits of contacts.
3.
Confidence
How
does your project demonstrate increased confidence among your
client communities? How could this be evaluated?
We
are aware of the constraints that the member organisations
work within and the anxieties and concerns that they had at
the start. We can compare these with the feedback at a series
of events we have held and the steadily increasing usage amongst
members and the mutual support they have given and the initiatives/innovations
in training they have developed for themselves and the language
they now use.
4.
Competencies
How
does your project increase community competencies? How could
this be evaluated?
The
project sets out to build the competencies of organisations
and individuals within them; the organisations are service
providing, campaigning or networking organisations working
with their own members and users. Increased skills etc within
the organisation impact on service users etc. Some organisations
are enabling users to learn ICT skills, setting up courses
etc. Evaluate by comparing start point with now; with feedback
from member
organisations.
There has been a formal external evaluation of the project.
5.
Capacity building - self-reliance and 'ownership'
How
does your project demonstrate community capacity building
— defining this as self-reliance and 'ownership'?
Women's
organisations often have a self help, development dimension
and are run by women from the community; greater communication
and transparency through internet usage can increase community
capacity.
6.
Choice
How
does your project increase and enable community choices? How
could this be evaluated?
Strengthening
women's organisations who serve/are run by a range of women
in the community, the social excluded (definition?) enables
community choices. Self evaluation by organisations about
the difference using the internet has made would be possible.
7.
Content
How
does your project increase the relevant content available
to community groups and enable communication to be expanded
and enhanced? How could the contribution of content be evaluated?
Organisations
are enabled to create their own web sites and can discuss
best practice. Their web sites have presence on the women
connect website, which is creating a resource for women's
organisations to access web sites; we could count/ask about
visits/hits on web sites.
7
‘C’s Interview with The Children's Society
Methodology
7
‘C’s pilot questions and criteria sent online for
online completion, and supplemented by telephone interview.
About
The Children's Society
The
stated objective of the Children's Society is to promote social
justice and social inclusion - particularly for children and
young people. It is a national organisation with 90-plus Projects
across the country. Tony has therefore sought to differentiate
between the corporate level, and 'Project-based' initiatives,
of which he has personal knowledge. Both staff and clients
can be seen as participants. He has therefore sought to relate
the questions both to the experience of staff (and volunteers)
and to the children, young people and communities | 
