DEVELOPING SAFETY ASSESSMENT SYSTEMS USING EXPERT SYSTEM SHELLS-1

Acknowledgements: The author wants to thank XpertRule Software LTD and mr. Tim Sell for being able to try Decision Author- the main software in which this prototype shall be built.

GENERAL ASPECTS

Safety domain of research is by excellence a domain based on expertise. Textbooks and theoretical knowledge are good but the safety expert which inspects three times a day a certain part of an enterprise is the ultimate safety dealer here.

A lot of expertise is transformed into lessons learned- that are used for training and improvement of existing safety attitudes. On the other part, this expertise could be also valued in order to build optimal and effective safety assessment systems.

An expert system is software that emulates the decision-making ability of a human expert. In our case- the expert part should interrogate the specific employees regarding safety aspects of an enterprise.

The next figure illustrates how a safety expert, with the necessary knowledge into the problem could improve a safety assessment process. The safety manager of a specific enterprise which performs a safety assessment will navigate through the safety assessment items, will assess something (the current safety situation inside his enterprise- preferably) and will develop a safety improvement plan- this being the outcome of the assessment. of course that this assessment could be external and then we will have third party auditors- but essentially the process is the same. We need from our safety expert:

-to design optimally and efficiently the safety assessment items- so that the answers of the safety manager should capture the real, most significant and important from the safety point of view aspects;

-to develop the objective and optimal safety assessment procedure so that a very subjective process should perform as objective as possible;

-to build a reliable and cost sensitive improvement plan that will do exactly this- to improve safety and health on the basis of the performed safety assessment.

Generally, the existing safety assessment methods are using textbook based evaluation items- in order to assure a very large usage possibility. This is good- on one part- because they are giving us a certain referential that is the same in USA, France or Romania. This is bad, on another part- because it is not using the existing safety expertise in defining and developing the safety assessment items.

In our paper we are proposing an original approach regarding the development of optimal safety assessment solutions using expert system developments.

Figure 1 Safety Expert helps Safety Manager to perform an objective assessment

Safety Knowledge Elicitation

Ok, we have tons of safety knowledge. Do we need more? What is in into knowledge elicitation? Expertise is a collection of knowledge, experience, skills, techniques, facts, rules and so on, and using them to lead to goals. This is also valuable for safety expertise. We must say that expertise is unique reported to the context, to the domain and also to the place where expertise is established. The knowledge is elicited chiefly from experts in the field and data/ information available from published literature. There are various known knowledge elicitation techniques available. The choice of technique to be used in the knowledge elicitation process depends on the nature of the situation within which the knowledge is elicited, the domain knowledge and availability of experts. Elicitation could be defined as the process of acquiring knowledge from a domain expert- with some specific problems. The knowledge engineer (Bech-Larsen, 1999) will find it difficult to structure the interviews. On one hand interrupting the expert may break his train of thought and lose useful information; on the other hand, along digression into anecdotes or irrelevant information will make the transcript difficult to understand. However, it is hard to judge what is irrelevant, or where the expert train of thought is leading. The problems are compounded by the shades of colour in language. The expert will use technical terms, and the knowledge engineer must make sure that he and the expert have the same understanding of this terminology. The words such as big, small, frequently, sometimes are fuzzy, it is difficult to define. Also, the definition may depend on the context. It may be that the expert himself is not even consistent in his use of words (Vanden, 2008).

Laddering (Reynolds, 1988) is a structured questioning technique derived from the repertory grid technique, a, specific knowledge elicitation method that could apply well into safety. In Laddering, a hierarchical structure of the domain is formed by asking questions designed to move up, down, and across the hierarchy... Laddering is enabling a hierarchy of concepts to be established. The common generic means-end chain, therefore, consists of Safety Attributes (SA), Safety Consequences (C) and Safety Added Value (SAV).

Safety Attributes -> Safety Consequences -> Safety Added Value

Laddering (Grunnert, 2005) allows crossing over from qualitative data gathering to quantitative data treatment. The laddering data analysis process (Jans 2006) typically involves the following two phases. First, the laddering interviews are transcribed and core elements -attributes, consequences, and values- are coded. This phase relies on the qualitative research tradition, requiring skills such as axial coding to define elements that emerge from the interviews. Once that the core elements are defined and labelled, the individual ladders can be decomposed based on these codes.

Laddering is illustrated in the next figure.

Figure 2 Laddering as safety knowledge elicitation technique

20 Questions is a method used to determine how the expert gathers information by having the expert as the knowledge engineer questions- having as output a. amount and type of information used to solve problems; how problem space is organized, or how expert has represented task-relevant knowledge.

In our research we have built a questionnaire for identifying, capturing and using heuristic knowledge, considering that we could add significant amount of knowledge to the existing one in the already done assessment. An example is given below.

New Knowledge elicited	Elicitation trigger	Former Knowledge
You must check especially if a maintenance job is repetitive done on the pipes that are alimenting processor X. Maintenance teams are usually solving punctual problems- without checking the safety of the whole structure. Be especially careful if they used to heat the pipes in order to make the maintenance.	From your experience- could you tell us something that could jeopardise the safety regarding alimentation pipes for processor X?	Alimentation pipes shall be checked.

Decision Author is well suited in order to introduce”Knowledge on the fly” into the expert structure, knowledge derived from elicitation.

The elicitation process is presented in detail in the next figure.

Figure 3 Elicitation process

To be continued...

HOW TO DEVELOP A BETTER SAFETY CHECKLIST

Latest developed tool :

HOW TO DEVELOP A BETTER SAFETY ASSESSMENT

Some images from the material...

RESEARCHING FOR A BETTER SAFETY ASSESSMENT-part 3

WHAT ARE WE GOING TO ASSESS?

There are a lot of checklists for safety assessment- some of this could be found also on Internet. A systemic approach to the safety assessment would however focus on general components- that could be developed by the safety assessor taking into account the specificity of the assessed workplace.

We have chosen- on the basis of our experience- a system with three main components:

-HUMAN RESOURCE- as humans are the root cause of more than 75% of the occupational accidents- this component is a must for every assessment. It should include- in our opinion:

- Technical training involved in the performance of specific tasks; was such training done properly and in time- taking into account not just the technical details but also the capacity of understanding of the employee?  Is this training updated when needed? Is this training enough?
-General and specific safety training;

-Employee experience in work and also at workplace;

-Employee abilities connected with the tasks to be done;

-Employee history of accidents and incidents;

-Team working abilities;

Part of these attributes are used and presented below in the model developed for this paper.

-EQUIPMENT AND MACHINES

-MANAGEMENT

The following table (Table 1) shows the data collection format used for the primary safety assessment.

The 10 attributes were chosen in the idea to assess various workplaces- with different activities. The auditor could change these attributes- taking into account however to have a unitary structure for the plant assessment.

The place of each attribute could be found in the figure...

The theory behind the assessment is considering that each workplace could be described by:

-its human resource, including also experience at work;

-Its technical resource- equipment’s and machines;

-its managerial resource;

A well trained human resource, with people selected taking into account their abilities in relationship with the tasks to be done; with the necessary experience- could be the greatest asset for a safe workplace.

A workplace should be designed having safety as one of the main goals. However, if this was not done in the beginning it could be done- partly- during the “active life” of the workplace. Re-designing for safety is one of the win-win solutions.

A workplace should have as much safety devices in pace as needed- considering the specific activities that are done there. The assessor should verify that the safety device is in place- functional- and that it is used accordingly with the safety goals. Some safety devices could be tampered with- in the idea to increase production by cutting safety paths.

Personal Protection Equipment’s should also be available in connexion with the safety design of the workplace and also with the safety devices in place. The last stand before an incident/accident should be the PPE.

The third component of the assessment should be the (safety) management of the workplace. A good manager would always include safety in his/hers management. However, a very good safety manager would assure commitment- from all the human actors at workplace, would also assure the necessary supervision and the maintenance activities that should be done on a daily basis (including cleaning, waste disposal, etc.) .The safety culture is developing in time- as the ideal link between employees and managers in assuring safety- anyhow these three attributes should be pursued from the management and assessed accordingly.

Figure 1.Place of assessment attributes in the whole image

 

ASSESSMENT AND EVALUATION OF THE RESULTS

To illustrate the possibilities of safety assessment and some instruments that could be helpful in the activities that are following the analysis and the interpretation of the results- on the basis of the previous graph we have developed a simple checklist- that could be re-developed as much as you like. It has (in this demo) 10 questions and the primary assessor- the one which goes to the workplace to perform the assessment- evaluates each of the questions on a 1 to 5 Likert scale.

Table 1

No	Item (Question)	1	2	3	4	5	Motivation
	HUMAN RESOURCE
1	Task specific abilities of the employees in the workplace (please consider the main task and all the tasks that are important from the safety point of view)
2	Training (we are considering here the professional training)
3	Safety training (including all the given safety training and its implementation into the workplace)
4	Experience at work- although this is a debatable item, experience at work has its importance to keep the employee out of  risks action
	EQUIPMENTS AND MACHINES
5	Safety design of the process, the afferent activities and the workplace- we recommend to ask for plans in order to have a full image
6	Safety devices in place, functional and that are known by the employees
7	Personal Protection Equipment (PPE)
	MANAGEMENT
8	Supervision of the employees
9	Maintenance of the workplace (including clean-up)
10	Commitment of the employees and their supervisors to safety (if there is a safety culture then it should be evaluated too)

As you can observe- there are general items (questions) that could be changed, modified or replaced- taking into account the workplace that is being assessed. However, once modified for a specific workplace- the checklist should be saved and used as a referential for the future assessments of this workplace. Eventually- a copy of the assessment checklists should be saved and sent also to the Competent Authority- so that the assessments that should be accepted and controlled by the Competent Authority could have a certain referential.

For the interpretation of the primary results we are using a software named ””Expert Choice”- that could be downloaded for free from Expert Choice Inc. Using this software we would try to optimize the assessment and make this assessment as objective as possible.  

Figure 2 shows the defined decision tree that has as its goal the safety assessment. On each branch there is an item.

Figure 2. Decision tree

For our demo we have considered a fictive”Manitoba Enterprise”; we worked with 5 workplaces to be assessed accordingly to the field results for each item. It is possible to make a direct evaluation- as seen in figure 3.

Figure 3 Direct evaluation

For a specific branch- like ”Task specific abilities” we are evaluating the five workplaces and are using a priority coefficient that is based on the results of the field assessment. As seen in figure- the first workplace performs the best for this item.

We could also compare the performances of two workplaces- like you can see in figure 4.

Figure  4 Comparision between two workplaces considering a specific item

It is also possible to perform the comparative evaluation by using a questionnaire to fill in the data.

Figure 5 Evaluating the results by the usage of a questionnaire

The software offers the possibility of a synthesis of all results like in the next figure.

Figure 6 Synthesis of the results taking into consideration all the items

Various graphs regarding the sensitivity of the evaluation can be obtained, like in the next figures.

Figure 7 Graph showing the performance sensitivity for all the nodes

We could also obtain a dynamic sensitivity.

Figure 8 Graph of dynamic sensitivity

Figure 9 Graph for gradient sensitivity

We could also compare head to head two nodes- like the Workplace 1 and Workplace 4.

Figure 10 Head-to head comparison

A global report could be obtained at least.

Figure 11 Global report preview.

RESEARCHING FOR A BETTER SAFETY ASSESSMENT-part 2

GOOD ASSESSMENT VS. FAILED ASSESSMENT

How can we make a clear distinction between a successful assessment of safety at workplaces and a failed one? The next table gives some indications- that are just orientated. Anyhow, you must consider that a safety assessment gives you very seldom an absolute result.

Table.1 Good vs. failes assessment

Good assessment	Failed assessment
It gives you the necessary information to make specific decisions and to develop improvement plans.	It is very categorical. It could be excellent- and in this case you should be very cautious- because nothing is excellent in industry- or could be very worse.
It gives you a risk level that you could use in benchmarking.	Gives a very impressive figure (regarding the risk level) that could not be used, says nothing and could be contested.
Includes all the technical details needed.	Approximates a lot, not including specific activities that are done at the workplace. This non-inclusion is a proof that the assessors are not very expert in their job.
Gives you preliminary reports regarding the workplaces being assessed and a final, consolidated report	Gives a generic report

It is possible to give a mark to the safety assessment job done by the external assessor, using the following table. If you are satisfied with the item- give 1- if not- gives correspondingly between 0 and 1.

Table 2. How to assess an assessment ?

Item	Maximum	Actual mark
1. Componence of the assessment team- no more than 6 persons if the enterprise is SME	1
2. Competence of the assessment team- at least 1 certified assessor with minimum 15 assessments done. At least one of the team members should be an engineer. If you have specific processes at least one member of the team should be an expert regarding those processes	1
3.Accreditation of the assessment company and the assessment team with the Public Authorities like the Work Inspection .All of the work being done in a safety assessment should be compliant with the Public Authority rules and should be certified (when needed) by them	1
4. Existence of safety assessment written procedures- that could be eventually be completed taking into account the characteristics of the assessed workplaces	1
5. Level of information and explanation given by the assessment team to those from the assessed workplaces that are involved in the assessment	1
6. Level of information and explanation given by the safety assessment team	1
7.Exhaustive check-lists for the main objects at workplace	1
8. Accurate description of the situation at workplace assessed- as to motivate the assessment	1
9.Logical and easy to use formula and notation for the global risk level	1
10. Correlation of the safety assessment results with the opinions of employee representative and top management	1
Total

For example- if the safety audit performed by the external firm is not approved by the top management, or by the employee representatives- there are some serious problems in the assessment and even if it is re-developed there should be given here 0 points.

Safety assessments with less than 7 points are highly contestable and the company should be asked to re-work or to mitigate the vulnerable points.

THE SAFETY ASSESSMENT TEAM

Depending on the complexity of the workplace, the primary safety assessment team could be composed by 1-6 persons.

Larger teams could be used for complex facilities. However, they became difficult to manage.

There should be always a team leader- generally the best assessor or the external assessor if we are working in a mixed team.

The team leader- or safety auditor as he appears in figure 1- should have as the main qualities:

A. a high level education- preferably in engineering;

B. at least 5 years working as safety assessor;

C. at least 15 safety assessments conducted by himself/herself;

D. good management qualities;

E. to be certified by a recognized organism;

F. capability of working long hours;

The difference in composing assessment teams is given by the complexity of the job to be done. The next table is showing this aspect.  The number and the participants are the results from our experience in safety assessment.

Table 3. Safety assessment team according to the type of workplace

Workplace type	Description	Safety assessment team
Simple	1-2 workers. Basic technical  equipment	1 person-  the safety auditor (external)
Medium	3-10 workers. Technical equipment with a reasonable degree off complexity.	2 persons. The safety auditor + one employee from the workplace that has the best experience and is safety committed.
Large	More than 10 workers and/or equipment that are very hazardous  with risk not just over the workplace but also on the facility and outside the facility	3-6 persons. The safety auditor +employee+ supervisor or line manager+2 experts +stakeholder

The safety auditor is essential in the assessment. However, our experience shows that when   the workplace is medium or large- in order to assure the maximum objectivity of the assessment- it is the best solution to add more members to the team.

When the workplace is medium-> large- the presence of one or more employees with the best working experience could be very helpful in assessing safety.

When the workplace is definitely large- the assessment team should be also completed with the supervisor/line manager that is directly responsible for the assessed workplace. Beyond the knowledge of the experienced worker the supervisor could give some hints about the quality of the management for this specific workplace. The continuous interaction between the safety auditor, the employee and the chief of the workplace would also lead towards a more efficient and objective assessment and also towards quick development for plans of improvement- if these are needed. A complete schema of the safety assessment team is given below.

Here also are finding places the experts in the field of activity plus a representative of the stakeholders. The safety assessment is not exactly a secret- so the participation of more people could improve (if managed well) the process of safety assessment.

The ideal team- from an external assessor- is presented in the figure 1.

Figure 1. The external safety assessment team

A mixed team- with one safety assessor that could be external- but not necessary- is presented in figure 2. 

Figure 2 The mixed safety assessment team

The figure shows that in the final assessment- the mixed team could be completed with 1-2 experts- generally technical experts that could give advice regarding the equipment used, the safety devices and the facility that is assessed and also by a representative of the stakeholders- if the enterprise has such stakeholders. The presence of the stakeholder representative is very important if the safety audit /assessment show that there are important changes to be made and serious resources should be involved.