Friday, October 09, 2009

My last examination

Today I told my fellow educators for the course COMP3410 at the Australian National University that this was my last examination. What I did not expand on was that this was not just for this course, I will not set any examination questions for any course. My hope is to able to continue to contribute to courses at ANU and elsewhere, and to design assessment for them, but do not ask me to write examination questions.

Last year I decided to give My Last Lecture and put into practice what I have been learning about combining online teaching and live group work. This was not an easy decision and has taken considerable work, both in reformulating course material and explaining the change to my colleagues.

My first ANU course with no lectures and no examinations finishes in a few weeks time and so far has gone reasonably well. Reformulating the material did take some work. What surprised me was the amount of explanation I had to do to colleagues. Many had difficulty with the idea of a course with no lectures. Even after I had announced that I would not be giving any more lectures there were many who asked me to give lectures. It took some time to reassure them that there were workable alternatives to lectures and this was permitted under university rules.

What I found surprising was that the formal university processes had no problems with a course with no lectures or examinations. Provided what was being taught and how it was being assessed was clearly spelt out, there was no impediment to a course without lectures or examinations.

The group who did not need any convincing were the students, who were very happy with the idea of no lectures. Some students seemed to think no lectures meant no work and an easy course, but soon changed their view when they saw there was compulsory assessed tutorial work each week. Some dropped out early, but not as many as for a conventional course.

I don't like setting examinations to be conducted on paper, over several hours, without the student being able to use external resources. This is not an effective form of assessment, nor is it useful for learning.

An examination is very different to the ways the students could expect to use what they are learning the real world. What is more realistic is have them briefly reply to a few short questions, discuss issues in a group and to write reports over days or weeks, with a library and the Internet at their disposal. That is the form of assessment I have used in the Green ICT course, with assessment for weekly discussion and written assignments.

There is a role for tests in courses and this can be by interactive assessment. These can be used to asses the student's level of knowledge to help with the learning as well as to give them a final grade. But the assessment should be a realistic facsimile of what happens in the real world. Assessment can use simulations and other techniques for being more realistic. Tests can be online and in part multiple choice. There can be many short tests, rather than a few large ones. I hope to explore this next year.

Labels: , , , , ,

Tuesday, June 09, 2009

Assessment and Teaching of 21st Century Skills project

Assessment and Teaching of 21st Century Skills project aims to develop globally standardised ICT-based assessment for school education which emphasise the use of ICT. The project came from a Call to Action by Cisco, Intel and Microsoft in January 2009. An example used in the call has students using the web to help solve a problem. What will be a bit disconcerting for education traditionalists is the spelling mistake "poplation" in the example.

It would be easy to dismiss ATC21S as self interest by computer companies wanting to sell computer hardware and software to schools. What might be reassuring (or not) is that the project is headed by Professor Barry McGaw at the University of Melbourne, with other academics from the University of California, Berkeley, University of Szeged, Hungary, University of Washington,and University of Toronto.

ATC21S has Working Groups on:
  1. 21st Century Skills
  2. Methodological Issues
  3. Technological Issues
  4. Classroom Learning Environments and Formative Evaluation
  5. Frameworks for New Assessments

Labels: , , ,

Sunday, December 14, 2008

Interactive Presentation on Carbon Footprint

A "flowgram" interactive "Presentation on Carbon Footprint Project-Basic Understanding" is available. This uses the
Adobe Flash Player to provide audio synchronised with web based content in the web browser.

Unlike some video based tools, Flowgram shows the actual web page in a frame in the web browser, not just an image captured of the page. In this case the narrator describes carbon footprint while pages from the US EPA are displayed. Text from the web page is highlighted as the narration continues. You can pause the presentation, rewinf and fast forward like a video.

You can also click on links in the displayed web page to break out of the confines of the prepared presentation. However, if you click to go elsewhere it is difficult to find your way back to the presentation. When you do get back to the presentation, it starts from the beginning, not where you left off.

Labels: , ,

Friday, September 12, 2008

e-Assessment makes the grade

Professor Geoffrey CrispThe University of Canberra is having a Learning & Teaching Week, 9-12 September, 2008 with free events, which non-UoC people, such as myself were invited to. One of the last sessions was the seminar "e-Assessment: More than just a grade", with Professor Geoffrey Crisp, Director of the Centre for Learning and Professional Development, University of Adelaide and author of the "e-Assessment Handbook".

The e-Assessment Handbook By Geoffrey CrispI attended a similar seminar by Professor Crisp, at ANU last October. The presentation has been updated with new material since then. He showed simple ways to implement computer assisted assessment using existing tools such as the Moodle and Web CT learning management systems, in conjunction with web based simulations and other interactive applications. There are demonstrations of the technquies available online (you need to create your own account for acceess). He also demonstrated the use of Ken Taylor's world leading VotaPedia system, which we used at the Open 2020 Summit.

This was a very useful and skillfully presented seminar, which combined high level educational theory, painlessly combined with practical tips for teaching. About the only negative was that the University of Canberra's technology was not quite up to the job, with the system controlling the video projector crashing and other audio visual problems impeding the presentation. However, Professor Crisp was able to skillfully work around these and use them as lessons about e-learning.
e-Assessment: more than just a grade This session explores some of the opportunities offered by online assessment to improve student outcomes and the quality of the assessment tasks. The work is based on the Carrick Associate Fellowship project and describes how academics can prepare interactive, computer-based assessments using helper tools such as browser plugins, java applets, QuickTime VR and Flash. The aim of the project is to assist teachers to move beyond simple multiple choice questions in an online environment to provide much richer, authentic and meaningful assessment tasks for students. Discipline examples may be viewed at

Labels: , , ,

Wednesday, October 10, 2007

Interactive e-Assesment

The e-Assessment Handbook By Geoffrey CrispProfessor Geoffrey Crisp, Director of the Centre for Learning and Professional Development and Professional Development and Director of Online Education at the University of Adelaide, presented a seminar on "Assesment: more than just multiple-choice" at ANU's CEDAM center in Canberra today. He showed a simple way to implement computer assisted assessment using existing tools, supported by research. I have been doing some of this with Moodle for commercial courses, but did not consider to be academically respectable. Now I know it is okay, I can use it for university courses. ;-)

Geoff started with an image from a medieval painting of a lecture in a monastery. He pointed out that apart from the lack of women, much of the way a lecture was presented then is much the same as now. The lecturer was at a podium at the front of the room, the students were in rows, some listening, some reading, some talking.

Geoff went through the types and purposes of assessment (it is not just because you need to pass). One use is to see what the students already know. "Clickers" can be used to get a quick assessment of the students knowledge in class. But more complex electronically collected assessment need not be marked automatically, it can be used just to collect the answers for human marking. However, before setting assessment the teacher needs to decide what scheme is appropriate and what is being assessed.

e-Assessment can have advantages for immersion and interactivity, social interactions. The results can look like computer and role playing games.

There are some examples of research on interactive, some beter than others, such as Literature Review of E-assessment (Futurelabs) and "Effective Practice with e-Assessment" (JSC).

Geoff showed an approach where the interaction is separated from the assessment. In this model the question invokes the interactive component. As an example the question is asked in a web page in a learning system, such as Moodle, this then invokes an interactive application provided via a web browser plugin and when the plugin is finished the student is returned to the web page where they can enter their answer.

What I found confusing was that this did not seem to be "interactive assessment", in that there was no interaction between the system asking the question and the application the student uses to research the answer. The computer presents a tradition text question, launches the interactive plugin for the student to interact with and then asks them for a traditional answer. There no assessment in the interactive environment. Geoff pointed out that this makes it a lot easier to create and is more flexible, but it took me some minutes to work out that this could be considered a legitimate "interactive" assessment.

Geoff pointed out the advantages of this approach: the academic can use pre-prepared modules and ask different questions about it for different levels of students. What also appears an advantage is that this only a small step beyond the computer based tutorials I prepare for ANU courses and is essentially the same as the system I used for a commercial training course.

Interactive Tutorial Example

With the ANU tutorials, a web page is prepared which looks much like a traditional set of notes for a tutorial/lab. However, the student is provided with a hypertext link which takes them to an online tool which they use to carry out part of the work. They copy some parameters from the tutorial sheet into the tool and copy some results back as their answer. An example is questions one to three of the metadata tutorial for "IT in E-Commerce" (COMP3410/COMP6341). In this the student is first asked to hand code some metadata, then use an online tool and compare the results and reflect on the differences. It would not have occurred to me to describe this approach as "interactive".

Interactive Assessment Example

With "Writing for the Web", a one day course prepared for a local government agency, there are a series of modules consisting of a set of slides used by the presenter live in the classroom, with accompanying notes for the students to read, followed by an exercise for them to do. The slides, notes and exercises are implemented using Moodle. The assignment module of Moodle was used to implement the exercises, but with no expectation that the results the students, just as a way for the student to type in the answers.

As an example in "Search the web for useful words", the student is asked to conduct search using two web tools and compare the results. They type their answers into the Moodle assignment web form (only visible to students enrolled in the course).

Interactive Assessment for Public Service Course

The Electronic Document Management module I am preparing for Systems Approach to the Management of Government Information was going to use the same style of online exercises as the writing fore the web course. However, I had assumed that I would need to provide a paper based test as well. But Professor Crisp's seminar seems to show that the interactive assessment could be used at least for part of the assessment. What would also make this easier would be purpose built flexible learning rooms for this form of teaching.

Postgraduate Online Professional Education for ICT

The ACS provides online courses in IT management using a Moodle based system for its Computer Professional Education Program. ACS also provide a Diploma of Information Technology for budding IT professionals in the Asia-Pacific region. e-Assessment could be of value for these.

More Interactive Assessment

However, to me having a web page launch a web application and then ask the student to copy result from it into a web form is not really "interactive assessment". Geoff commented that the assessment can be built into the interactive application, but this then reduces the flexibility. However, a level of interactive could be created by the assessment system passing parameters to the application and the application passing back some results at the end. This could use Web Services. It could be as simple as the assessment system adding some parameters to the end of the URL used to invoke the application. Results could be returned using XML encoded data, as is done for elelctronic tax returns, or even just a web page with the results included with Microformats. This would need some work, as my one attempt at including microformats in Moodle, the Moodle web editor modified the content breaking the Microformat.

Next Gen Not Wired

Geoff ended the seminar by cautioning that "Next Gen" students will not necessary benefit from, or want to use, social networking tools for education, even thought they use them socially. So educators should not assume the students will want to do everything via a computer, mobile phone and iPod.

See Also
  • Centre for Learning and Professional Development's moodle system.
  • Books on e-Learning
  • "The e-Assessment Handbook" by Geoffrey Crisp, Continuum International Publishing Group, 2007 (available from Amazon.Com):
    Assessing learning in an online environment is being used by teachers and institutions at an increasing rate. Learners are demanding a more flexible approach to assessment activities just as they have done with learning. This demand will increase as online practice becomes embedded into all schools, further and higher educational courses and corporate training programs. As students are engaging with content in an online environment, they will also need to be assessed using the same tools that are used for the learning experience. This book provides practical guidance to various aspects of online assessment including:
    • types of assessment
    • choosing the right software
    • examples of e-assessment over a wide-variety of disciplines
    • making e-assessment interactive

    Table Of Contents Preface
    Section 1 How does assessment affect learning?
    1. Overview
    • Why do we assess?
    • Relationship between learning and assessment
    • Assessment for learning
    • Assessment and grading

    2. Guidelines for assessment practices
    - Professional bodies publishing guidelines on assessment
    - Assessment models
    - e-Assessment practices and their relationship to learning
    3. Types of e-Assessment items
    - types of e-assessment
    4. Cosequences of using e-assessment
    - Risks involved in using e-assessment
    - How is e-assessment currently being used?
    - What are the limitations of e-assessments?
    5. Choosing the right e-assessment software
    - choosing an appropriate assessment format
    - how do you want to administer your e-assessments?
    - what do you want to administer your e-assessments?
    - what type of questions do you want to ask?
    - choices in e-assessment software
    - other types in online assessment software
    - adaptive assessment software

    Section 3 What does e-assessment look like?
    6. e-assessment examples by discipline
    - General sources of e-assessment questions
    - Accounting, Commerce and Economics
    - Arts and Humanities
    - Biological and Health Sciences
    - Chemical Sciences
    - Earth and Environmental Sciences
    - Physics and Engineering
    - Mathematics

    Section 4. What are the practical issues when doing e-assessment
    7. Item and Test Bank use
    - constructing Ttem and Test Banks
    - using Item and Test Banks
    - non-adaptive test delivery methods
    - adaptive test delivery methods
    8. Grading and marking Items from the Test Bank
    - general grading and marking issues
    - negative and confidence level marking of objective items
    - marking selective response items
    9. Validity and reliability
    - overview of validity and reliability
    - classical test theory
    - Item response theory
    - Rasch modelling
    - estimating reliability
    - evaluation of test banks
    10. Standards, specifications and guidelines
    - assessment standards
    - national standards
    - professional association standards and guidelines
    - identifying standards
    - content standards, specifications and guidelines
    - assessment content metadata
    - interoperability
    11. Security issues
    - authentication of appropriate access
    - secure assessment environments and software
    - standards and guidelines
    - what to do when things go wrong
    12. Accessibility issues
    - statutory requirements for accessibility
    - good practice guidelines
    - testing accessibility
    - improving accessibility
    - assistive technologies
    Section 5 How Do I Make e-Assessment Interactive
    13. Assessing discussion groups and collaborative tasks
    - communicating and collaborating online
    - overview
    - examples of software for e-collaboration
    - design principles
    - collaborative problem solving and group work
    - online discussion groups
    14. Interactive assessment and Java applets
    - learning designs
    - simulations
    - Java applets
    15. Assessing role playing and games
    - characteristics of online role-plays
    - senario-based
    - game-based
    Section 6 What about the future?
    16. The future of e-assessment
    - current perceptions of where e-assessment is heading
    - drivers for change
    - what will it be like in the future

Labels: , ,