Analytical Capacity: Experience from Practice

INTRAC’s Analytical Skills Training Programme in Central Asia23
Shaped by the recent Soviet past and influenced by traditions of hierarchy and conservative attitudes, Central Asia is a region prone to conformist thinking. The formal education system provides little encouragement for critical thinking and, during Soviet times, scientists were expected to produce positive research results in support of the official political discourse. The same word is used in Russian for research and analysis and the prevailing understanding of research emphasizes the use of statistics and quantitative data.
Within this context civil society is still in its infancy. This has meant that CSO leaders have limited recognition or influence in society and are often sidelined by government officials, despite the fact that the majority of these leaders have previously served the Soviet government system in managerial positions. Furthermore, few CSOs have emerged from a genuine constituency and most of them therefore struggle to relate to the community groups which their missions suggest they support. In the light of these circumstances, strengthening the skills of staff to understand, analyse and influence policy was identified within INTRAC’s Central Asia Civil Society Strengthening Programme as a key issue for the organizational effectiveness of CSOs in the region.
From 2002–2004, INTRAC conducted an Analytical Skills Training Programme (ASTP) in three countries: Kyrgyzstan, Kazakhstan, and Tajikistan. The programme aimed to help local CSOs engage more effectively in policy influencing by building their capacity to conduct socio-economic analysis. The original focus of the training was to build research skills, but this evolved based on the recognition that many CSOs needed analytical skills and the capacity to think critically. Participants attended five modules over nine months which were structured as follows:

                      INTRAC Analytical Skills Training Programme: Course Outline
Module 1 presented an overview of development theories, and used the Six Thinking Hats exercise to illustrate that there are different ways of analyzing issues. Guidance was given on developing research questions to rigorously describe (‘what?’, ‘how?’, ‘why?’) and propose hypotheses for examination (‘what if?’).
Module 2 presented different interpretations of the concept of poverty, explored the emergence and practice of social development and social impact assessment as a method of inquiry, and provided the opportunity to practice participatory tools of data collection.
Module 3 examined participatory methods and supported individuals in the development of their own research questions and methodologies. Participants were to carry out the majority of their research before module 4.
Module 4 examined the value of monitoring, evaluation and learning systems. It explored approaches to analyzing data (including ways of combining qualitative and quantitative techniques) and supported individuals in their own analysis. In addition, methods of disseminating findings were introduced.
Module 5 took the form of a mini-conference. Participants presented the results of their individual research projects to representatives from the CSO sector, media, academic institutions, and local and national government bodies who participated in feedback discussions on research findings and methods.
In 2005, interviews were conducted with selected ASTP training participants and trainers to obtain an understanding of the perceived and experienced impact of the training. While the primary purpose of the training was to build individual capacity to influence policy, it seems that the knowledge and skills gained can also lead to strengthened organisational capacity if participants are able to share their acquired competencies with colleagues and influence the ways of working in their organisations. INTRAC has also reflected internally on the lessons learnt from implementing the ASTP. The combined areas of learning identified by these reflections are summarised below:

Thinking Creatively

Organizations tend to interpret their own realities by defining both the area of work they will focus on and how that work will be conducted. The danger is that this may limit the organization to old and ineffective ways of working and/or to conforming with received and perhaps inappropriate models. The development of analytical capacity is a way of fostering alternative ways of interpreting the world, which perhaps fit better with the reality observed. An important example of how such a ‘paradigm shift’ in development thinking has affected the ways agencies work is Amartya Sen’s work on freedoms, entitlements and capabilities22, which shifted the emphasis of human development from a concentration on economic growth to recognition of wider influences on human well-being.
Real insights can be derived from careful consideration, sharing of realities and creatively rethinking the frameworks which describe these realities.
At individual or organisational levels, there can be similar ‘paradigm shifts’ where real insights can be derived from careful consideration, sharing of realities and creatively rethinking the frameworks which describe these realities. For example, indigenous coffee producing co-operatives in Mexico were vulnerable to the fluctuation of world coffee prices and to the exploitative prices paid by the middlemen, or coyotes. However, with the help of external advisors, one co-operative began to analyse the situation differently. As a result of a market analysis they realised that they could reduce their vulnerability and increase their income by by-passing the middlemen and finding new markets for their coffee. This led to the establishment of a very successful chain of urban coffee shops which increased the income received by the co-operative by 1) adding value to the coffee beans and 2) creating a national market for coffee that had barely existed previously.
                             Creating Meaning: Section Summary
In order to interpret information, organizations need to reflect critically on the values and assumptions of different actors, start to explore the larger patterns and creatively rethink existing frameworks if these prove to be inadequate. 

Interpreting Patterns

A pattern is a regular and repeated series of events. Analytical processes involve recognising and interpreting these patterns as they emerge. In this way an organisation can begin to make sense of multiple pieces of information. Deriving patterns starts by reviewing and summarising the information collected. This can then be compared and contrasted with the original conceptual framework used. The validity and reliability of the evidence can again be questioned and contradictions identified. Trying to make sense of the information prematurely may lead to false conclusions or to the omission of important information. This stage in the process therefore requires the application of methods for collecting, processing and organising information which are flexible enough to permit reorganisation as additional information confirms or questions existing interpretations.21 This may reveal a need for further data collection to resolve such contradictions.
By recognising and interpreting patterns organisations can begin to make sense of multiple pieces of information.
Where it is apparent that the information does not fit with existing models or frameworks these may need to be revised or conceptualized. Information can then be shared more broadly within and outside an organization, to determine whether there is broad agreement about the picture presented. The relevance of existing models or analysis can subsequently be discussed, as well as the possibilities of developing new models.
While the process of interpretation described can help organisations to make sense of the information they collect, there may be considerable resistance to questioning
established ways of looking at, and interpreting, the world. In some contexts, for example where there is a strong patron––client culture or a recent history of conflict and trauma, people may be reluctant to question or challenge the status quo and feel more comfortable interpreting the world in a way which is familiar and safe. In these situations it is often more appropriate to take a longer-term approach that encourages people to interpret their reality using their ‘wisdom’ or ‘judgement’ rather than falling back on pre-defined ‘answers’. 

Understanding Analytical Capacity

Part 3
Reflecting Critically
Once a rigorous description of the various realities has been institutionalized or given value, the next stage in analysis is to critically reflect on the information and descriptions that have been collected. This can involve questioning the evidence, for example by asking:
• How accurate is the information?
• Which methods were used to collect the information?
• Are there gaps or additional information which would be useful to have?
• Whose knowledge has been included or excluded?
• Whose perspective does the information represent?
• Might the information be biased towards particular perspectives?
In addition, the underlying assumptions and values of those involved need to be recognized and articulated before conclusions are made. Assumptions are the unstated ideas that we accept to be true or take for granted without necessarily having evidence to support them. Values can be described as the ideas or beliefs that people see as worthwhile, meaning those that provide standards of conduct by which we measure the quality of human behavior.18 Both assumption and values can be very influential in guiding people through a process of reflecting on and interpreting information. However, they can also be potentially deceptive because people may not be aware of the way in which their values and assumptions are directing their reasoning and, consequently, their conclusions.
Since the development sector focuses on processes of social transformation across many different cultures and contexts, the implicit values and assumptions of the different actors involved may vary considerably. If these values and assumptions are not discussed openly and made explicit, it may result in a conflict of values, one value being treated preferentially over another or, ultimately, the wrong conclusion
being drawn. There are many common, but often unproven, assumptions which some of those involved in development make – such as the preconceived idea that agricultural development will lead to increased incomes or that literacy makes for a more liberal society.
These assumptions and values can lead to inappropriate and/or ineffective development initiatives. For example, after 10 years of civil war on the island of Bougainvillea in Papua New Guinea a programme was established to support village development committees to set up income-generating projects. Traditional councils of elders were exclusively male so it was assumed that the only way to ensure the inclusion of women was for them to have their own committees and projects. However, many of the projects that were established, such as poultry production, were later vandalized as a result of jealousies between husbands and their wives. On reflection it was recognized that as the local culture was matrilineal, although women were not formally involved in village councils, they did have many informal ways of influencing decisions. Separating men and women had therefore inadvertently emphasized their exclusion rather than promoting their inclusion.
A lack of understanding of the local culture and context can lead to inappropriate conclusions being drawn from a process of critical reflection.

Understanding Analytical Capacity

Part 2
Becoming Aware of Different Perspectives
Observing reality involves gaining an awareness of the context and nature of the issues an organization is facing. However, there is also a need for organizations to understand how different groups of people and stakeholders perceive these issues. Trends in social development work have emphasized the use of qualitative research methods and participatory methods of data collection and analysis, giving testimony to the increasing importance attributed to understanding these different perceptions. In addition, the internal perspectives within an organization may differ depending on the diverse interests of different departments, groupings or individuals. The task then is to bring these different external and internal perspectives into the core of organizational understanding.
Opening the opportunity to explore the different viewpoints within an organization, and of its different stakeholder groups, can enable an organization to begin to observe and make sense of its own reality. This may involve taking the time to slow down and allow people to become aware of their own thought processes, or ‘mental models’, but also to explore other perspectives and ways of thinking. Recognizing that we each have our own mental models can help when exploring the way different individual perceptions influence processes of collective reflection within organizations (see text box below).

                                           Mental Models
‘Mental models’ can be described as deeply ingrained assumptions and practices that influence how individuals within any organization or group understand and interact with their environment. According to the organizational learning specialist Peter Senge (1990), such preconceived ideas are shaped by experience, social and cultural background, as well as the organizational context in which these individuals operate. These mental models act as filters for the selection and interpretation of information as it crosses organizational boundaries. Mental models therefore determine the ways in which the individuals within an organization analyses and interpret information and adapt organizational strategies.
The willingness to recognize mental models and flexibility to revise them within an organization can be nurtured by striking the balance between creating space for collective reflection and analysis but also providing clear direction and leadership. This can be a long and gradual process since it is often difficult for individuals or organizations to break out of their ‘mental trap’ or mindset.

Encouraging people to stand back from their habitual ways of perceiving the world and not be constrained by per-determined frameworks can also enhance an awareness of different perspectives. One practical tool that can help to recognize and explore different ways of thinking is the ‘six thinking hats’ methodology developed by Edward de Bono.

Understanding Analytical Capacity

Part 1
In the previous section we explained why we are focusing on analytical and adaptive capacities and suggested a process of organizational analysis and adaptation. In this section we begin to develop an understanding of analytical capacity and suggest that the purpose of being analytical needs to be linked more explicitly to improved actions or policies. Analytical capacities should include, but extend beyond, the ability to reflect on practice and analyses data. They should also involve the ability to look at issues from new perspectives, to think ‘outside the box’ and to develop new paradigms if needed. The process of investigation, of being analytical about our work, therefore requires a continuous process of conceptualizing and re-conceptualizing. This is not merely about gathering information according to per-determined frameworks and views of the world. It is also about detecting signs that these frameworks have become inadequate or less relevant, and searching for new ways of articulating or conceptualizing the world in order to resolve issues.
Developing analytical capacity involves encouraging organizations to observe their reality and to question received models, modifying them or changing them where necessary.
Organizations, like individuals, observe and interpret reality according to previous experiences which may then be fitted into received/accepted ways of viewing the world. There may be certain areas of work where organizations primarily build their knowledge base from their cumulative experience of carrying out regular activities – such as delivering services. However, in identifying new areas of work or reviewing strategic directions, the tendency may be for an organization to base their analysis on predominant models or ways of seeing things rather than thinking afresh. In the aid sector, social development work is complex and very context specific, and those who drive the agenda (e.g. donors) – however well meaning – may find that their current paradigms or models are too often accepted uncritically by development organizations/actors.
Developing analytical capacity therefore involves encouraging organizations to observe their reality and to question the received models, modifying them or changing them where necessary. This implies an evolving interaction between theory and practice, as is described in following sections, through:
Developing analytical capacity involves encouraging organizations to observe their reality and to question received models, modifying them or changing them where necessary.
• Observing reality by articulating the view of the world that we hold, and understanding this from different perspectives (concepts).
• Creating meaning by selecting ways of collecting, reflecting on and interpreting information in relation to specific situations or issues (data collection and analysis). This new knowledge can then be fed back into the views we hold about the world (back to the concepts again).
Observing Reality
This first step in the analytical process is an exercise in imaginative description – that is describing a specific situation in detail and as accurately as possible, before proceeding to creating meaning from the information gathered. It involves standing back from reality to see things with fresh eyes and not getting too stuck in habitual ways of perceiving the world. Due to the complex nature of social development, and the dynamic interrelationships involved, developing a view of the world can involve starting to see the world as a ‘whole’. 
Understanding the Whole
Organizations, and the people within them, should be seen not just as passive observers of the world but as active participants which are intricately interconnected with the wider system in which they operate11. Conventional analysis has tended towards simplifying complexity by reducing the whole into a series of constituent parts. This reductionist analysis can be a logical way of thinking about machines, which operate as a whole when all the parts are assembled, but is less useful for the dynamic human systems which characterize organizations because of its limited ability to capture:
• the dynamics and interrelationships within an organization as a ‘living’ system which is constantly growing and evolving
• the relationships and interactions between the organization and the complex social, political, economic, cultural and physical environment in which it operates.

Linking Analysis and Adaptation

Linking the strengthening of analytical capacity with the development of adaptive capacity can provide a basis for ensuring that an organisation has the judgement, self-confidence and time to critically assess a situation and to use this learning to improve actions or strategies. If one happens without the other it may lead to 1) improved understanding but no subsequent relevant action or 2) ill-informed adaptation which responds reactively, and potentially inappropriately to a given situation. Analysis and adaptation should therefore be perceived as integrally linked and interdependent10.

Within an organisation the process of analysis and adaptation involves the capacity to:
• Observe reality: by standing back to ‘see’ and collect information about the complex internal and external environment of which it is part.
• Create meaning: by reflecting critically on this information and exploring different perspectives to develop a new understanding that is unconstrained by established frameworks.
• Sense emerging events: by identifying wider patterns and events within the environment, whether predicted or not, and deciding which of these might require, or ‘trigger’, a response.
• Adjust actions: by making improvements to the approach used, experimenting with innovative ideas and exploring alternative solutions.
While these may take place simultaneously it is suggested that consciously carrying them out as a cyclical process may be more effective. The more integrated and continuous the processes become, the more proactive the organisation can be in its response because it is able to monitor, reflect, and act upon triggers for change. The learning gained from these experiences can then be fed back into future analysis. This is illustrated in the diagram below.
Process of Organizational Analysis and Adaptation

 

Adaptive Capacity

Adaptation is a process of adjusting to new conditions to become better suited to the context or environment. Adaptive capacities can enable CSOs to improve their actions as a proactive response to changing circumstances, i.e. to co-evolve with their environment. Adaptation can be described as a process of assimilation and accommodation9 where through:
• assimilation, new experiences and knowledge derived from events in the environment are incorporated into existing ways of thinking about the world.
• accommodation, existing ways of thinking are revised according to the new knowledge and lead to new actions or behaviour.
This dual process of assimilation and accommodation, often triggered by change, instability, conflict, or crisis, can enable individuals and organisations to link analysis with adapting the actions they take. Adaptive capacity can be defined as:
                      Adaptive Capacity (Action)
The capacity to strategically adjust thinking and actions in response to changing circumstances based on relevant knowledge and improved understanding

Adaptive capacity includes the ability to:
• Use improved understanding to adjust actions appropriately
• Expand awareness to sense new and emerging events
• Respond proactively to changing circumstances
• Have insights into intended and unintended consequences of action
• Continuously observe and correct course of action
• Draw lessons from changed actions and integrate them into future analysis

Analytical Capacity

Analysis can be described as the detailed examination of an object, organization or system in order to interpret or explain it8. While cognitive sciences can sometimes
give the impression that analysis focuses solely on breaking the whole down into its constituent parts, for the purpose of this paper we argue that it should also include the process of understanding the whole. In this way the analytical capacity of an organisation relates to its ability to examine and understand its internal and external environment and the interactions within and between them. This can be about establishing cause and effect but also about understanding patterns of behaviour and evolution. Analytical capacity can be defined as:
                Analytical Capacity (Understanding)
The capacity to observe the whole, identify patterns, reflect critically and understand dynamics and interactions, while remaining open to new ideas and perspectives

Analytical capacity includes the ability to:
• Stand back to observe and reflect on the internal and external context
• Understand larger patterns, dynamics and interrelationships
• Look at issues from many angles and viewpoints
• See beyond established ways of thinking
• Attribute meaning to information
• Identify root causes of success and failure
• Construct simple models to conceptualise understanding

Why Analytical and Adaptive Capacities?

The concept of analytical and adaptive capacities builds on existing practices such as strategic planning and organisational learning. Originally developed for military purposes strategic planning deals with the overall identity, direction, goals and objectives of an organisation. This model, based on a rationalist economic approach, is now being criticised as too reliant on pre-determined outcomes, a ‘knowable’ environment and a predictable future3. Planning, deciding, monitoring and controlling the ensuing process may be all that is needed in situations where change is relatively straightforward and easy to predict4. However, ‘when you are facing very difficult issues or dilemmas, when very different people need to align in a very complex setting, and when the future might be different from the past, a different process is required’5. Those CSOs involved in development are often engaged in processes of social transformation and are therefore inherently facing complex situations and trying to create a future which is different from the past. There is therefore a need to explore different ways of conceptualising strategic thinking, action, reflection and learning to better reflect the open-ended and unpredictable nature of development.
The process of analysis and adaptation described in this paper constitutes a non-linear, flexible approach which recognises that organisations operate in complex environments within which they have many dynamic interrelationships. The process is closely linked to the theory and practice of organisational learning as a developmental process that integrates thinking and doing6. Building the capacity of organisations to analyse and adapt can enable them to consciously and purposefully:
• reflect on and make sense of their own reality;
• recognise themselves as an integral part of the whole, i.e. as dynamically interconnected with the wider environment;
• move beyond pre-determined frameworks and habits of action7;
• respond proactively to events as they emerge; and
• influence and create alternative realities.
But what do we mean by analytical and adaptive capacities?