Overview of the Assessment Landscape
For a talent assessment client or practitioner it is important to have access to information about the talent assessment landscape when selecting a solution for your company or practice
The talent landscape can be considered from a number of angles. Firstly, the types of tests available; secondly, the different systems; and thirdly, the current providers.
Talent Assessment Types
The first thing any would-be user of talent assessments should be aware of is the range of constructs that can be assessed. Often the issue of talent and what it is, can be underestimated and over-simplified. In pursuit of ever shorter assessment times and simple models that are easier to explain to line managers, companies often come up with almost unethical solutions.
Evalex research into the dimensions that drive individual performance, career success, high performance teams and high performance companies, indicates that a wide holistic approach should be taken and therefore a wide range of tests needs to be included in any assessment battery.
Unlike other companies, Evalex does not use a simplistic “one-construct” test to make decisions about promotions or appointments. Other companies may not be aware of it, but should a candidate that has been declined on this basis take them on in a labour dispute, they could be in trouble. Be aware of test suppliers selling you a “one-test-solves-all” solution. Often these tests only measure personality or styles, which shows some correlation with performance but far less than a holistic test battery.
The diagram on the left indicates the various constructs that make up human functioning. It is important to ensure that an assessment solution comprises a cognitive test, a personality or styles test, an interest questionnaire and a competence test such as assessment centre simulations or technical assessments.
Talent Assessment Systems
Most systems come packaged with an assessment instrument. This can range from a manual process (no integration) of handing out test books and answer sheets, to logging into a website to do a test (semi-integrated), or logging into a talent system where a number of tests are completed on one platform (fully integrated).
For a proper overview of the distribution systems used to facilitate assessments, the total assessment value chain must be considered. We believe the value chain consists of the following process events:
- Design philosophy of the assessment tool
- Test choice in setting up the assessment
- Facilitation of the actual assessment
- Assessment scoring and norming
- Assessment reporting
- Feedback to decision maker and candidate
- Invoicing and billing administration
The ultimate system would manage these seven “moments of truth” in the value chain through a well-designed, integrated system.
The question is, which systems do this?
We have scanned the market and found that most companies’ assessment practices fall essentially into three models. :
1. Manual and semi manual (no integration): This involves administering a number of tests – some manual and some automated – characterised by logging into different systems.
Advantage:
• Easy for any consulting psychologist or in-house assessment function to set up
Negatives:
- Slow turnaround time and high cost of process
- Inconsistent design philosophies
- Labour intensive administration
- Site bound administration
- Frustrating and fragmented user experience, having to log into different test procedures
- Final integration of different test reports into one integrated report is labour intensive and highly dependent on the competence and experience of the psychologist
- Billing administration is convoluted and inefficient; each candidate’s assessment has to be matched to different supplier invoices and credits
2. Test aggregators (semi-integrated): Test aggregators have developed a computer system that aggregates a number of supplier tools, leading to one platform for administration.
Advantages:
- Three points of integration: One assessment experience for candidate, one linear report and one integrated feedback session
- Lower cost of process
- Faster turnaround time
Negatives:
- Incoherent design philosophies: As each test in the system may have been developed by a different test publisher, the design philosophies and psychological theories differ
- Inconsistent scoring and norming: Each test is standardised according to its own norm or population group. So the scores on dimensions have all been compared to different norm groups leading to inconsistent norming of talent levels
- Linear reporting: The report lists dimension performance in a linear fashion with little pattern detection
- Billing administration is slightly better than the manual/semi-manual process, but still convoluted
3. Evalex (fully integrated): At Evalex we have developed our own assessment platform AND our own psychometrics.
Advantages:
- Six points of integration
- One design philosophy across all tests
- Assessment instruments cover a full range of human constructs: Few suppliers offer value questionnaires, interest assessments, technical competency assessments and assessment centres (management simulations) in addition to the normal cognitive and personality battery
- System generated pattern detection report: A good psychologist will detect patterns; a high scoring dimension in one test linking to a low scoring dimension in a second test linking with another high scoring dimension in a third test indicates a pattern that could be interpreted as a strength or development area. The Evalex software has been programmed to detect patterns for:
- 1. Strengths
- 2. Development areas
- 3. 48 performance inhibitors
- 4. 17 archetypical roles in organisations and how the candidate matches them
- Quality control: Our sophisticated reporting engine ensures that all patterns are detected; quality becomes system guaranteed rather than dependent on the psychologist’s skill
Talent Assessment Providers
Assessment instruments are provided by a range of practitioners. Four types have been identified, including independent psychologists, test suppliers, large consulting firms, test aggregators and fully integrated solutions.
The chart below indicates the business model of each, with the associated advantages and disadvantages.
The choice is entirely up to the client and all four models provide solutions. For small companies that do not employ their own psychometrist or psychologist, the independent psychologist can work well by providing expertise on an ad hoc basis.
Assessment firms such as large auditing and consulting companies will be more suited to larger companies, where volume and throughput are critical. Test providers are useful where a particular range of tests need to be sourced for a project.
The market is, however, definitely moving towards talent assessment system providers (test aggregators) simply for ease of use and lower cost of process. Many test providers are now working with system providers who offer tests through their systems.
When it comes to talent system providers, it makes sense to use the most integrated system.
Evalex Talent Assessment System
Evalex presents one of the most advanced and integrated assessment systems on the market.
The Evalex Talent Assessment system was conceived in 1980 and has evolved over time through research and application. It was first written into a computer application in 1986, with revisions in 1992, then enhanced as an Internet-based assessment tool in 2000, with constant upgrades and expansion. It is now one of the most advanced Internet-based assessment platforms on the market.
What makes the Evalex Talent Assessment System unique and differentiates it from others available on the market is the incorporation of four distinctly different assessment technologies for the assessment of cognitive, behavioural, leadership and technical competencies.
Most other systems focus either on assessing only behavioural competencies or management competencies. They therefore present either ability/personality/styles/motives questionnaires or only assessment centres, but are not strong on providing both or the inclusion of technical competencies.
Four Main Technologies
Evalex is the only assessment platform that integrates four major assessment technologies into one solution, namely:
- Assessment centre technology
- Psychometrics
- The Evalex Competence Inventory (business and technical competencies)
- Digital animated interactive exercises
Integration of Four Technologies into One Assessment Session
Ours is the only assessment platform where the psychologist can create an integrated assessment session combining six assessment centre simulations/case studies and multiple psychometric tools that the candidate can complete in one online session, providing a totally integrated report across all the tools.
What makes Evalex unique is that, based on years of experience and research, we have developed our own assessment technology consisting of a range of assessment instruments that can assess:
- Cognitive competencies
- Leadership competencies
- Behavioural competencies
- Business competencies
- Technical competencies
- Learning potential
The system can handle the following moments of truth in the assessment value chain:
Evalex Business Simulation
What predicts leadership potential best and how do we assess this?
Some of the most crucial decisions to made in the talent management space are as follows:
- Recruitment: Which candidate from a shortlist should we appoint?
- Promotion: Is a manager ready for promotion? Should we promote from within rather than appoint from outside?
- Development and succession planning: What development would a manager need to move to the next level?
Knowing how a candidate will perform based on interviews, CVs and reference checks is often a shot in the dark. Research has shown that these indirect techniques have a very low predictive validity. Imagine if you could immerse your candidate in the role on a trial basis before making any hiring or promotion decisions. Obviously this is not a practical solution. The next best thing, however, is to create a simulated environment for your candidate to operate in.
To resolve this issue, we first need to ask two crucial questions. Firstly, what is a true indication of leadership success? The second question flows from the first: Which assessment technique predicts potential and talent the best? Cognitive tests, personality inventories or management simulations?
This statement perfectly sums up the true indication of leadership success:
“When wrestling on a daily basis with competitor, market, client and staff challenges, it is natural to assume that, on balance, those who survive these skirmishes best will naturally escalate to more senior positions.”
The career track record of an individual, the progression from one level of work to the next level of complexity, combined with the staying power at that next level, only to be promoted to deal with an even higher level of complexity, may well be the ultimate measure of leadership success.
Leadership is about how effectively we deal with and manage key events. Situations we are presented with on a daily basis, and the accumulation of the incremental additional efficiency that great leaders add to every event, culminate in visibly distinguished high-performance organisation.
The solution
If true success is achieved by dealing efficiently with the situations and challenges in our environment, why rely on cognitive and personality tests alone (both of which simply indicate a propensity to behave in a certain way) when we can submit the candidate to a real-life simulation of the job s/he will have to perform? The solution is the Evalex Business Simulation (EBS). By immersing the candidate in a simulated management environment we measure actual competencies when it come to dealing with the typical issues s/he will face once appointed or promoted. We measure real behaviour, not hypothetical behaviour.
What does the research say?
In our insights section you will find research publications providing a detailed research results, but let us just consider the highlights of that research here.
Our own research indicated that leadership competencies assessed through EBS had the highest predictive validity, followed by cognitive, then personality and level of work tests. By predictive validity we mean the ability to accurately predict future performance from the performance in the assessments.
These findings correspond to two other major research studies, namely Sackett, Shewach and Keiser (University of Minnesota, 2015) and Woehr, Matthew and Fleisher (University of Tennessee, 2008).
From the analysis it would seem that leadership competence simulations identify the majority of variances in job level, no matter how you model them. Conversely, in only conducting cognitive and personality test, less than half the story of an individual’s potential is explained.
The graphic below explains the extent to which different assessment methodologies predict potential and performance.
The research provides sound evidence and justification why the Evalex35 Leadership Assessment solution consists of management simulation, cognitive and personality tests – the results of which lead to a balanced statement about an individual’s potential.
Clearly in assembling a management or leadership assessment battery, the foundational instrument, the starting point, should be assessment centre simulations, augmented by cognitive or levels of work assessments, and personality tests. Unfortunately, most companies mainly use the cognitive-personality combination.
Latest research findings put forward a strong case for assessment centres as more powerful predictors than cognitive and personality tests. This is leading to a shift in our understanding of what drives executive success.
EBS in action
Leadership must be one of the most critical challenges in any organisations and In developing business leadership, it is crucial to use the correct assessment approaches to identify talent in order to reap the best return on subsequent development efforts. Using a less than optimal solution in assessing candidates for managerial and leadership roles can waste valuable time and resources.
EBS is an online, cloud-based assessment centre that puts candidates through their paces in six management simulations to assess management competence, emotional maturity and thought leadership.
The EBS solution synthesises leadership competence, thought leadership, personality and styles (optional with Evalex Psychometrics), experience and performance into a talent classification and leader utilisation strategy. It is a powerful product that allows you to gather comprehensive talent intelligence on each leader in your company. This data is run through our advanced talent management software to provide a rich tapestry of information at both individual and corporate level.
At an individual level, EBS produces:
- A talent strategy for each leader
- Development plans for each leader
- Talent fit to current role
- Talent fit to future roles
- A development plan to move from current to future role
At a corporate level, EBS provides:
- Information on the quality and depth of talent at each level of management
- Identification of which levels of management are either at risk or conversely well stocked with talent
- Your company’s talent pipeline, benchmarked against other organisations locally and globally
- Leadership development priorities for incorporation into your corporate leadership development strategy
During the four-hour Evalex Business Simulation, the candidate is required to respond to around 30 typical organisational and business issues. The way s/he deals with these issues projects his/her competencies, thought leadership, emotional maturity, management style, business knowledge and experience.
Responses are analysed by highly trained and experienced organisational psychologists. The candidate’s performance is compared against position-level benchmarks based on the Levels of Work framework, allowing us to form a picture of the level and type of position where this individual should be optimally utilised.
Leadership situations
To be effective, a leader must be able to deal effectively with six critical management situations, all of which are dealt with in EBS.
Leadership in problem analysis and decision-making (digital animated interactive performance-based exercise): EBS present a business problem in an online environment where the candidate interacts with information in a dynamic way to assess problem solving and decision effectiveness competence.
Leadership in general management (inbox): Typical organisational and management issues are dealt with, such as procurement, process design, cash flow and analysis of financial figures.
Leadership in project management (business case): This involves translating strategic direction of the business into a project plan.
Leadership in staff management (caselets): This includes typical staff management issues such as performance, skills development, remuneration and unions.
Leadership in client relationship management (caselets): Here we look at typical client management issues including service delivery, client complaints and pricing.
Leadership in interaction management (role play) A number of business issues are dealt with in an interactive situation.
Evalex Psychometrics
Our psychometrics basket comprises a carefully chosen combination of psychometric instruments to provide high predictive validity solutions.
Through extensive research we have isolated the key drivers of performance at each level of work and developed psychometric instruments to measure each driver. All psychometric instruments have been developed by OMT and Evalex and are backed up by research to substantiate reliability and validity.
Assessment instruments are integrated into one assessment platform, which includes the IP200 offered in collaboration with Integrity International.
Assessment solutions
We offer a carefully chosen combination of psychometric instruments to provide high predictive validity solutions. Our solutions include a combination of assessment centre instruments, psychometric instruments and technical assessments.
Based on careful research we have developed solutions that we have found to predict performance at the different levels of work. Each level of work requires a different style and personality, a different level of EQ and a different level of cognitive ability and competence. In studying these levels we designed a battery of assessments that predict performance at each level.
Levels and Types of Assessment
For each level of work, a combination of instruments has been included, based on research indicating the drivers of performance at that level.
The power of using assessment tools from one system is the aspect of progression. It is important to note that the assessment tool that you use must be able to “hurdle” you into the next assessment battery as you pass through the employee life cycle.
Let us assume that we have employed an operator. She has completed the Evalex10 and she is performing well. We would like to promote her. Rather than re-assessing her, we merely add two additional questionnaires and she will then have completed the Evalex20. Two years later, if she shows potential for promotion, rather than re-assessing we add the four case studies of the Evalex30. The same applies to moving to the Evalex40.
All of this is done effortlessly and almost automatically in the system. What this means is that over the long term you have a reduction in cost of assessments.
This also has implications for the talent management process. If you use the standard battery of assessments, catering for all levels in the organisation, and all people in the business have completed the same set of assessments, you can compare individuals’ data against multiple benchmarks.
When to use which solution
The Evalex solutions have been designed based on the type of assessment required at each level of work. The competencies driving performance at each level were established through internal research. The competencies to be assessed, in turn, inform what assessment instrument is required.
Evalex5: Work level 0,5: The Odyssey Talent System has been designed exclusively as a one-stop-shop for the assessment of competencies typically required in entry level roles.Odyssey is a suite of six questionnaires assessing the following competencies:
- English literacy
- Numerical literacy
- Work process assimilation and execution
- Elementary problem solving
- Clerical efficiency
- Work speed and productivity
- Interests
- Learning Potential
Evalex10: Work level 1,0 to 1,5: At this level we need to find out more about:
- Cognitive ability at the level of work complexity: Mainly for learning and problem solving
- The required personality traits for the job
- Interest in the required work or tasks to be performed, giving an indication of motivation
Evalex15: Work levels 1,5 to 2,0: As these roles start moving into the team leader or technical domains, we need to assess:
- Cognitive ability at the level of work complexity, mainly for learning and problem solving
- The required personality traits for the job
- Interest in the required work or tasks to be performed, which gives an indication of motivation
- Preferred work styles
Evalex20: Work levels 2,0 to 2,5: As these roles start moving into the junior management and professional domains, we need data on:
- Cognitive ability at the level of work complexity
- The required personality traits for the job
- Interest in the required work or tasks to be performed, which gives an indication of motivation
- Preferred work styles
- Business acumen
- Values
Evalex30 to 40: Work levels 3,0 to 5,0: As these roles function at middle to senior leadership level, we need to assess:
- Cognitive ability at the level of work complexity
- Management style issues: The required personality traits, interest in the required work or tasks to be performed, preferred work styles, business acumen and values
- Leadership competencies
- Thought leadership
- Emotional maturity