SEQ

Develop one written question and one practical question in a field of your choice.

For each question you have constructed indicate
a. Outcome/s covered

b. Content area/s covered

c. Intended domain level (cognitive, psychomotor, attitudinal) and level on Miller’s pyramid

Critically analyze the questions that you have constructed highlighting their strengths and weaknesses
(You need to construct one questions from each category, i.e. one MCQ or SEQ and one OSCE station)

N.B. Reed all questions, allocated makes mention end of the questions.

Structured Essay Question
Mr. Sirisen a 62year old estate worker and heavy smoker came to the A & E department of a local hospital complaining of cough and shortness of breath for a 20 minutes period. He produced the Estate Medical Assistance Record which mentioned that he has not been treated for any medical illness during the last 20 years except for a cut injury of the hand. His pulse rate was 90/m. BP 130/85, auscultation of the heart was in not fund triple sound and murmurs. His ECG showed sinus tachycardia with RBS of 106 mg/dl and the pulsoxy meter reading was SPO2 88%.

1. List the 3 most likely causes for his SOB? (5 x 3=15)

……………………………………………………………………………………….
……………………………………………………………………………………….
……………………………………………………………………………………….
2. What is the next most important (non biochemical) investigation? (10)
………………………………………………………………………………………

3. The patent was kept in prop up position and oxygen given via nasal catheter and the IV cannula was inserted. His troponin level and dimer levels were analyzed by standard laboratory (private). All the findings were within the normal limits.
What could be the most likely finding of the investigation mentioned in question 2.

…………………………………………………………………………………………….. (10)
4. List 4 possible positive findings of his physical examination in favor of your diagnosis? (3x4=12)
………………………………………………………………………………………………………………………………………………………………………………………………
…………………………………………………………………………………………………………………………………………………………………………………………………

5. What is the best treatment option? (13)
………………………………………………………………………………….

6. What are basic items you need for your best treatment option? (05)
…………………………………………………………………

7. What are the common complications of your treatment protocol? (2.5 x4=10)
I……………………………………………………………………………….
II……………………………………………………………………………….
III………………………………………………………………………………..
IV……………………………………………………………………………….

8 Briefly describe how you would avoid one of the complications mentioned above? (25)
(Do not exceed the given space)

………………………………………………………………………………………………
………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………




















a) Outcome/s covered

Structured Essay Question - expected outcomes

Structured essay question has developed to expect the following outcome
1. Analytical ability of deciphering information (Symptoms, signs and investigation)
2. Understanding and recall of memory behind the Clinical scenario.
3. Application of clinical sciences skills appropriately
4. Synthesis and creativity in scientific writing


Students at the Final MBBS level should have analytical skills of deciphering complicated data in a clinical scenario. History examination and investigation findings should have been used for methodical analysis.
Then they could understand the theory concerning each piece of information and recall what they have learnt from memory to be applied to a different clinical setting they encounter. (Classroom, Ward) Such as what are the causes for sudden shortness of berth in middle aged men
Student could have the ability to apply facts to clinical settings through recall. For example, why they have mentioned that the d dimer test is negative and this is because it provides less possibility for pulmonary embolism.
Student should have the capacity to understand the exercise consisting of previously unseen related questions. In the future they may face similar incidents in real life. (Comprehension)
They would have the ability to synthesize and create new thinking using the existing information such as how to avoid complications of your treatment protocol.

b) Content area/s covered

Content area of the SEQ

Content area covers the following subject area.
Respiratory medicine
Cardiology
Biochemistry
Thoracic surgery
Emergency management

Respiratory medicine – The student should understand the area related to respiratory medicine, pneumothorax, pulmonary embolism etc.
Cardiology-Student should know about ischemic heart diseases.
Biochemistry-Should have knowledge to interpret several biochemical investigations.
Eg- Troponin
Thoracic surgery-Surgical procedures such as tube thoracotomy, complications etc.
Emergency management-Students should have knowledge about emergency management procedures.

Objective structured clinical Examination


A




B


1. Name A and B(5x2=10)

A………………………………… B ……………………………

2. Advise how to use A (The 45 year old mother is sitting in front of the table). (90)








OSCE expected outcomes

This question is based on checking skills and attitudes. But there should be knowledge behind the skill potion (Krathwohl, at el.1973).Part one of the question is based on memory recall. Part B is based on checking the behaviors of skills such as communication and establishing rapport with patients (attitude).
OSCE content area
This question has covered respiratory medicine. E.g. The inhaler technique.
Medical ethics - how they introduce themselves, how they describe the subject area concerned in a professional way and also verbal and nonverbal communication.


































C) Intended domain level (cognitive, psychomotor, and attitudinal) and level on Miller’s pyramid

Cognitive domain
In 1948 Benjamin s. Blooms introduced the Bloom’s Taxonomy for learning. IT consist mainly of the cognitive domain. After that there was the addition of two more domains and now there exists 3 main domains i.e. the cognitive, psychomotor and affective domains. (Krathwohl, at el.1973).


Structured essay question

Structured essay question has been designed for the assessment of cognitive and psychomotor aspects of clinical medicine
Cognitive aspect consists of 6 components. These are Knowledge, Comprehension, Application, Analysis, Synthesis and Evaluation.
This structerd Eassy question has the elements of all six levels
The first level is knoledge.This step expects theemember or recall of information such as facts, terminology, problem-solving strategies and rules. In this question, student has to engage more in this steps because it has posed a lot of question based on the theory of medical science. There are questions, lists and names mentioned in relation to the knowledge level. (Krathwohl, at el.1973).
The next level is comprehension. At this level there should be some degree of understanding exhibited in terms of what has been read or ability to perceive connections or relationships with the given picture .But it is not a major components of this question.
Application is another important step of this question. Students can apply theoretical principles to solve problems. Student can use respiratory medicine knowledge and apply likely courses for the SOB
Students can analyze the strengths and limitations of each of the useful theoretical approaches for understanding specific practical problems. The SEQ mentions what is the most likely diagnosis? What is the best treatment option?
Students can combine theoretical approaches to explain complex practical problems.
How would you avoid one complication? This is one question used for synthesis but there is big role at synthesis level.
The student has the ability to judge the value of material (statement, novel, poem, research report). Decisions are based on definite criteria’ Criteria may be internal (organization) or external (relevance to the purpose). This is the highest level in the hierarchy.
The students can select the theoretical approach that is most applicable to problem solving and according to that existing knowledge of pneumothorax they can provide the answer to the best available treatment plan (Krathwohl, at el.1973).

Objective structured clinical Examination

OSCE has been developed in relation to all 3 domains. It has more cognitive and psychomotor domains with less of the affective portion (Harden 1975).
At the knowledge level, the student should have to memorize about inhalers. Learning objectives at this level he/she should know common terms, specific facts, methods and procedures. Also, the student should memorize basic concepts, and principles(Krathwohl, at el.1973).Then student should understand the instructions given in the question and materials to justify method or procedure. The student should use inhalers to demonstrations with knowledge of its indications and to whom it’s being described. Next is the application level. The student is in new situation but he should apply concepts and principles to situations for problem solving.
Also the student can analyze the strengths such as what are available, limitations, and what are the limits in terms of time and the patient understanding.
Then student can synthesis a new practical approach to demonstrate the inhaler technique to the patient.
Student should show the ability to judge the value of material for a given purpose. Ex. Use of inhalers and related devices can beef effectively demonstrated.





Affective domain
The affective domain is the second domain of Bloom's Taxonomy. This domain deals with our attitudes, values, and emotions. It is the "valuing" domain. It deals with the student’s attitude and there are five levels in this domain (Krathwohl, at el.1973).



Receving is open for new information or experience from lesion.
Responding is active participation via interaction with or response to new information or experiences. Assessing of the student is also a method of increasing the effectiveness that is comes under the affective domain.
Valuing is worth or value a person attaches to a particular behavior. Behavior consistent with belief or attitude (Krathwohl, at el.1973).
This level involves forming a reason why and making appropriate choice between two things.
Characterization is concerned with control behavior by the value system. Students’ behavior should be pervasive, consistent, and predictable. This structured easy question does not concern the affective domain.

Objective structured clinical Examination
This question has to reach the higher level of the affective domain. It is checking attitude and incorporating new information into the existing value system at organization level. (Harden 1975). The student can be assessed on their behavior and attitude by this question while demonstration how polite and professional they are.









Psychomotor domain
Learning outcome is shown as a physical skills such as coordination, dexterity, manipulation, grace, strength and speed .This is introduced by Blooms as five levels initially and then developed by Harrow to six levels and finally seven levels by Simpson. (Harrow 1972).

Manual or physical skills comes under the psychomotor domain. . It is called the “doing" domain. It has six levels. These are imitation .manipulation, precision, articulation, and naturalization. This is called Harrow taxonomy (Harrow 1972).

Simple reflexes begin at the lowest level of the taxonomy, while complex neuromuscular coordination makes up the highest level.
Reflex movement
Level one reflex movement occurs without learning in response to some stimuli.



Basic fundamental movement
The next level is the basic fundamental movement In this step, the student observes and then imitates an action. The expected outcome is that the individual should be able to observe and then repeat an action.
Perceptual
In perceptional level should be performance of an action with written or verbal command without visual advice. It can be ability to reproduce action with control and minimum errors.
Physical activities
Physical activities level student should have the ability to produce sounds via the efficient functioning of the body. There should be a display of a series of acts that should be performed accurately with control and perfect timing.
Skilled movement
Skilled movement’s level has been designed for the acquisition of a degree of efficiency when the students are performing a complex task.
Non discursive communication
Non discursive communication ranges from facial expressions to complex movements.
Structured essay question developed at perceptual level student should have ability to reproduce with written instruction. All the written instruction can lead to reproduce an image in the clinical setup.
OSCE has been developed according to cater to all six levels. There are reflex movements while showing a patient without knowledge such as smiling.
Then it move to basic fundamental movement they observe in the situation and react accordingly. Ex- with given instrument they may think there may be a demonstration. After that they will read written instruction and try to work accordingly at the perceptual level. EX they will select the instrument and think on how to demonstrate it.
At perceptual level they could have to act according to written instruction with inhalers.
Then they have to start physical activities such as demonstration of the inhaler technique. It should be done within the given time.
Then they move to skilled movements according to what they have learned previously. At the same time they may show expressions (Body language) of the importance steps in using inhalers (Harrow 1972).




Levels on Miller’s pyramid






Structerd eassy question has been developed on the knows and how levels . Some questions like what are the instruments tally with the knows level and , most likely diagnosis, complications tally with knows how level ( Miller 1990).
OSCE has been developed in show how level of Millers pyramid. Students have to show how inhalers are used. That will show their skills . It is similer to real practice . therefore it can asses how they face real practice



















Critically analyze the questions that you have constructed highlighting their strengths and weaknesses
Assessment should be good in all aspects of reliability, validity, educational impact, acceptability, practicability, and legal defensibility and cost effective. (Ronald and Epstein, 2011).
The structerd essay question

The structerd essay question has been developed to check the expected outcomes related to knowledge, atitiude and skills.This is a good parameter because the MBBS is a profesional course. It covers all three aspects (knowledge, skills and attitudes).Also this question coversa wide range of content areas such as medicine, surgery and ethics.. This is a good sign because student should have broad knoledge .Students have a chance to answer at least a few part on his favourite question. When considering the evaluation process of question such as validity, reliability, practicability, and legal defensibility and cost effective
Reliability of the assessment is the reproducibility of the same result in several occasions. But SEQ reliability is low compared to the MCQ. In this question there are practical steps, students will not write the same answer each time.
There are four principal ways to estimate the reliability of a measure (Ronald and Epstein, 2011).
1. Inter-observer: Deferent examiners agree to give the same rating. The paper or other performance. In this SEQ there are some parallel answers , there is a chance to deferent examiner give deferent marks and reliability will be low
2. Test-retest: Same test gives same result at two different times.
SEQ will not give the same answer with the repeat test
3. Parallel-forms: the same subject is examined using two different measurements and comparable results are given. This does not happen in this SEQ
4. Split-half reliability: Here it compares half of a set of test items with the other half and determining the extent to which they yield similar results but because of a single question it will not affected.
The following factors can affect reliability: (Lambert at el, 2010) of this question
When the length of assessment is long, it will produce more reliable results. (Ronald and Epstein, 2011).But this SEQ is not too long and reliability will be low.
The suitability, of the questions or the test for the students is being assessed.
The phrasing and terminology of the questions. It is not bad most of the time when simple terminology is used.
Administration of the test – Advice is given before the test such as time, writing skills etc... Time limit and amount of marks for each part has been mentioned. Thus that student can understand to give more attention to more marks questions
The design of the marking schedule and moderation of marking procedures
SEQ does not give much difficulty to the designed marking schedule
. But structured essay questions can be completed within 20 minutes.

Miller G.E. (1990).
Validity is an important good evaluating tool of assessment. Validity is measuring to what extent assessments have tested the students. Actually it should done without contamination with other characteristics (Dent and Harden 2013). Any assessment covers the curriculum of the course. The entire curriculum cannot be tested in the assessments. But it should test the most important areas of outcome in the curriculum. According to Millers pyramid the higher level of the pyramid is more valid than the lower level. A good test should have high reliability and validity. ( Miller 1990).
Face validity is place for to check that assessment items appear to be appropriate. In this exam there are several components that looks appropriate. (Ronald and Epstein, 2011)
Criterion related validity is covering concept, skills and knowledge. As this is a medical science course skills assessment part should be more .SEQ will assess knowledge more than skills. There for it is a negative point in SEQ
Content validity is the most useful validity of this examination. Content validity is giving the content and it must be accurately represented by the test items. (Ronald and Epstein, 2011).
In this SEQ has a high level of content validity as it has covered a vast area of clinical medicine that should be completed at MBBS level.
Educational impact of the assessment is another important tool to improve this course.. This SEQ has an educational impact as it is based on more practical problems.
Fairness of an exam refers to its freedom from any kind of bias. (Lambert at el, 2010). The exam should be appropriate to all qualified examinees irrespective of race, religion, gender, or age. In this type of SEQ will help to minimize examiners bias
Acceptability of the question SEQ is an accepted method.. Feasibility of the structured essay is high, Aspects in practicability of the assessment with SEQ is not unsatisfactory.
Cost of the examination is low in SEQ. Program to have legal defensibility that means the test’s quality would stand in a court challenge. This SEQ developed on MBBS level with standard and therefore it has legal defensibility.



The Objective structured clinical Examination

Objective structured clinical Examination should be more objective than subjective. Similar marks should be given for each student .Also it should be structured to cover range of the MBBS curriculum. Student should apply clinical and theory knowledge both in this question (Lambert at el, 2010).

This OSCE has developed to check knowledge, skills and attitude. But it has more skill portion. This has simulate real clinical set up. Skills such as communication skills, clinical judgments, demonstration and subject knowledge about inhalers. Communication skills like verbal, listening, and nonverbal and language skills can be checked. Student should has to listen patient, and he has to tell how this inhalers are using, he has to show it demonstration and non-verbal gestures. (Harden 1975).
This OSCE has covered more practical respiratory medicine problem as one in eight population may need inhaler treatments. Within 3 –to 5, minute student can demonstrate if they have trained adequately. This is better than traditional clinical examination, because OSCE ability to control its complexity and can mentioned clearly about skills, attitudes, and problem-solving capacity (Lambert at el,2010).

This question can be evaluate with reliability, validity, educational impact, etc...
Reliability can increase with number of questions.

1. Inter-observer: Deferent examiners agree to give the same rating. In this OSCE there are deferent way to demonstrate and patient education therefor , there is a chance to deferent examiner give deferent marks and reliability will be low
2. Test-retest: Same test is given the same result in two deferent times. But this type SEQ will don’t give same answer when repeat test even in same student, there for reliability will be low.
3. Parallel-forms: same subject is examined by using the two different measurements and comparable results are given. This is not happening in this OSCE
4. Split-half reliability: Here comparing half of a set of test items with the other half and determining the extent to which they yield similar results. Because of single question .It will not affect.
Then we consider validity, according to miller’s pyramid OSCE is in higher level of pyramid. It is more valid than lower level MCQ, and Essay. OSCES have high reliability and validity (Miller 1990).
Face validity is place for to check that assessment items appear to be appropriate. In this exam question there are several components it is looks like appropriate in respiratory medicine, communication, ethics, patients education. (Ronald and Epstein, 2011).
Criterion related validity is covering concept, skills and knowledge. As this is Medical science course skills assessment part should be more .OSCE will assess more knowledge than skills. In that wives this is a good question.
Content validity the content and it must be accurately represented by the test items.
In this OSCE has high level of content validity as it has covered vast area of clinical skills should be complete in MBBS level.
Educational Impact of the assessment is another important tool to improve in this course.. This OSCE has educational impact, as it is based on more practical problems, in future they will face similar problems while they are doing professional practice.
Fairness of an exam refers to its freedom from any kind of bias. (Lambert at el,2010). The exam should be appropriate for all qualified examinees irrespective of race, religion, gender, or age. In this type of OSCE will have examiners bias, because this is a demonstration, and communication part is prominent?
Acceptability of the question OSCE is an accepted method. But Feasibility of the OSCE, is not high as a practical procedure, Aspects in practicability of the assessment with OSCE is lower than Essay and MCQ, as this type of question there should be, instruments, Simulate patient actual patient, and examiner .Because of high requirements practicability will be low.
Cost of the examination is high in OSCE as it has high requirements but it has high cost effectiveness when conceder future out come
Program to have legal defensibility that mean test’s quality that would stand up in a court challenge. This OSCE developed on MBBS level with standard, according to curriculum. There for has legal defensibility.
Broth question covered adequately their expectation but there may be a limitation with time. That can be avoid reducing number of question appropriately.




References
Dent,J.A.,Harden,R.A.(2013) curriculum development ,Assessment , practical Guide for Medical Teachers,4,pp3-52,285-289.

Harden.R.M.,Stevenson,M.,Dowing.W.W.,Wilson.G.M.,(1975) Assessment of Clinical Competence using Objective Structured Examination, Medical Education., British MedicalJournal, 1975, 1, 447-451
Harrow, A.J. (1972). A taxonomy of the psychomotor domain. New York: David McKay Co.pp.8-9
Jones,K.O, Harland, J,JulietReid, M.V. and Bartlett, R.(2009), Relationship Between Examination Questions and Bloom’s Taxonomy , ASEE/IEEE Frontiers in Education Conference, pp-2-6.
Krathwohl, D. R., Bloom, B. S., & Masia, B. B. (1973). Taxonomy of Educational Objectives, the Classification of Educational Goals. Handbook II: Affective Domain. New York: David McKay Co., Inc.
Lambert ,W.T.,Schuwirath and Vandervleuten,P.M.(2010), How to design a useful test, The principles of Assessment ,Understanding Medical education.pp198-206.

Miller G.E. (1990); The assessment of clinical skills/ competence/ performance. Acad Med 65:s63-s67.
Ronald M. and Epstein, M.D.(2011) Assessment in Medical Education ,Medical Education , The New England Journal of Medicine pp.387 -396