Assessment Rubric: Build-a-Bot Activity
| Level 1 | Level 2 | Level 3 | Level 4 | |
|---|---|---|---|---|
| Knowledge | ||||
| Knowledge of content (e.g., concepts of Large Language Models, training vs. programming) | demonstrates limited knowledge of content | demonstrates some knowledge of content | demonstrates considerable knowledge of content | demonstrates thorough knowledge of content |
| Understanding of content (e.g., comprehension of AI’s purpose and potential for bias or hallucination) | demonstrates limited understanding of content | demonstrates some understanding of content | demonstrates considerable understanding of content | demonstrates thorough understanding of content |
|
Thinking |
||||
| Use of planning skills (e.g., generating ideas for the bot’s role and gathering necessary instructions) | uses planning skills with limited effectiveness | uses planning skills with some effectiveness | uses planning skills with considerable effectiveness | uses planning skills with a high degree of effectiveness |
| Use of processing skills (e.g., analyzing risks and identifying appropriate guardrails for the role) | uses processing skills with limited effectiveness | uses processing skills with some effectiveness | uses processing skills with considerable effectiveness | uses processing skills with a high degree of effectiveness |
| Use of critical/creative thinking processes (e.g., creating a unique bot persona and reflecting on risk mitigation) | uses critical/creative thinking processes with limited effectiveness | uses critical/creative thinking processes with some effectiveness | uses critical/creative thinking processes with considerable effectiveness | uses critical/creative thinking processes with a high degree of effectiveness |
|
Communication |
||||
| Expression and organization of ideas and information (e.g., clarity and logic of the prompt and reflection) | expresses and organizes ideas with limited effectiveness | expresses and organizes ideas with some effectiveness | expresses and organizes ideas with considerable effectiveness | expresses and organizes ideas with a high degree of effectiveness |
| Communication for different audiences and purposes (e.g., establishing the correct voice and style for the bot) | communicates for different purposes with limited effectiveness | communicates for different purposes with some effectiveness | communicates for different purposes with considerable effectiveness | communicates for different purposes with a high degree of effectiveness |
| Use of conventions, vocabulary, and terminology (e.g., correct use of terms like “hallucination,” “guardrails,” and “LLM”) | uses conventions and terminology with limited effectiveness | uses conventions and terminology with some effectiveness | uses conventions and terminology with considerable effectiveness | uses conventions and terminology with a high degree of effectiveness |
|
Application |
||||
| Application of knowledge and skills (e.g., applying the components of Role, Purpose, Qualities, and Guardrails in the prompt) | applies knowledge and skills with limited effectiveness | applies knowledge and skills with some effectiveness | applies knowledge and skills with considerable effectiveness | applies knowledge and skills with a high degree of effectiveness |
| Transfer of knowledge and skills (e.g., applying red-teaming insights to a new bot context) | transfers knowledge and skills with limited effectiveness | transfers knowledge and skills with some effectiveness | transfers knowledge and skills with considerable effectiveness | transfers knowledge and skills with a high degree of effectiveness |
| Making connections within and between various contexts (e.g., connecting AI behavior to ethical impacts on society) | makes connections with limited effectiveness | makes connections with some effectiveness | makes connections with considerable effectiveness | makes connections with a high degree of effectiveness |