new testing tool ready by 12-07

 

ETUDES notification digest for Mar 1, 2007

 

1. Mneme: Test Center (new testing tool by Dec. 2007)

2. Overview of new Delivery component - Mneme: Test Center

---

 

1. Mneme: Test Center (new testing tool by Dec. 2007)

 

From: "Vivie Sinou" Date: Thu, 1 Mar 2007 22:16:23 -0800To: Subject: Mneme: Test Center (new testing tool by Dec. 2007)

Dear Colleagues:

 

Since last October, Glenn Golden, Software Architect (U of Michigan) and I

(Foothill/ETUDES) have been working on the delivery component of Tests &

Quizzes. The plan was to refactor the delivery module of the Tests & Quizzes

tool so that it can perform well under heavy load and handle concurrency

(simultaneous testers) - its major weakness.

 

The goal was to have this work done and put the new code onto our ETUDES-NG

servers over the Christmas break. However, shortly after we started, Glenn

determined that the code had serious problems and the delivery module (the

side where students take the assessments) needed to be *completely*

rewritten. Refactoring was not an option.

 

So, this became a 5-months project. Mneme: Test Center, a brand new delivery

module, was born. Mneme, a name selected for the new tool/code, by Glenn, is

the muse of memorization (appropriate for testing), and the sister of

Melete. Mneme will replace the delivery component of the Tests & Quizzes

tool, possibly as soon as April. We'll run the other components of the Tests

& Quizzes tool (authoring, grading, and pools), side-by-side with Mneme for

now.

 

In addition to great performance improvements that we hope to see in

delivery (pending load and performance testing later this month), I saw this

as a great opportunity to fix known usability issues with delivery and some

design flaws. Since this was going to be a complete rewrite (not a single

line of code from T&Q), it made no sense to not fix the issues and put them

in! I didn't want to change too much, however, so that the transition is

easy for students.

 

I will send you a list of the improvements that you will see next month in a

separate email shortly.

 

A great development (though stressful decision) of this work was the

realization that we needed a new testing tool. We focus on distance learning

and administer over 20,000 tests and quizzes a year. We use testing

extensively. Although T&Q has stabilized significantly since we integrated

it with ETUDES-NG two years ago, it still has design/usability flaws that

are unforgiving. The recent reports of its flaws with randomization is what

did it for me. It was time to move on.

 

After lengthy discussions in the past month, the ETUDES Project and U of

Michigan have decided to partner and complete Mneme - Test Center, a brand

new testing tool. We will keep the features and functionality of T&Q that we

know teachers like and want, but we will fix the usability issues and design

flaws that have been problematic (i.e. inability to allow resubmissions and

have to publish new tests, inability to change settings after a test is

published, inability to have true randomization, and the list goes on and

on). We still need to resolve funding for this new development. However,

even if we don't raise additional funding, we'll STILL COMPLETE Mneme: Test

Center. We will have to table work on other tools for later. My team and I

are over-extended already so this will add more on our plate for the next

12-16 months, so I ask you for your patience as we table other features and

focus on completing Mneme.

 

The goal is to have the T&Q tool *out* of our ETUDES-NG servers with the

December 2007 upgrade. We believe it's doable. I will send you more details

on the plan and will keep you informed of our progress as we make

strides....

 

Thank you for your understanding and support.

 

Best,

Vivie

 

--

Vivie Sinou

Dean, Learning Technology

ETUDES Consortium Project

Foothill College

 

 

 

 

---

 

2. Overview of new Delivery component - Mneme: Test Center

 

From: "Vivie Sinou" Date: Thu, 1 Mar 2007 22:28:13 -0800To: Subject: Overview of new Delivery component - Mneme: Test Center

Dear Colleagues:

 

In addition to performance improvements that we hope to see in the delivery

of tests on the student side of things (pending load and performance

testing), the following issues were addressed in Mneme: Test Center /

Delivery.

 

Please note that although Glenn Golden (U of Michigan) and I are 90% done

with this work, slight changes may still occur in the next 15 days, leading

up to the 2.4 code freeze, as we refine some things and get feedback from

faculty and students during QA and usability testing. But, I wanted to keep

you informed on our progress and work up-to-date.

 

Again, we hope to put this new code in production in early April. We will

run it alongside Tests & Quizzes (seamlessly), until we have the rest of

Mneme: Test Center done in December, 2007, at which time, we'll retire T&Q.

 

As always, I will keep you informed on our progress. Let me know if you have

any questions.

 

Best,

Vivie

---

Overview - Mneme: Test Center / Delivery

---

 

Navigation

---

'Finish' is included on every page - not just on last question and table of

contents (TOC). A navigation bar is included at top and bottom of all pages

of a test (linear or random order). All buttons are included in each page,

but if not used (as in linear order where there is no 'previous' and no

TOC), they are grayed out.

 

Rationale

---

Rationale has its own icon in the legend of the table of contents. If

learners have left the rationale box empty in a question, this is now

highlighted in the TOC.

 

Finish Exam (submit for grading)

---

'Finish' has two behaviors:

 

• If a learner has left questions unanswered, upon clicking on 'Finish', we

take him to TOC to review and complete what's not answered, review what's

marked for review, and fill in incomplete rationales. Learners can click on

'Finish' and this will submit the test after a single confirm - 'press

again' confirmation request.

 

• The other finish, available from any question and the TOC, is when there

are no unanswered questions, no questions marked for review, and no

rationales left blank. This will submit the test/answers after a single

'press again' confirmation.

 

Basically, learners can finish a test when done reviewing/answering

questions from *anywhere*. Even if questions are unanswered, after a single

confirm upon clicking on Finish, they can submit a test for grading, if they

want.

 

Auto-Submitting Tests with Hard Due Dates

---

Mneme treats assessments with hard due dates (i.e. no late submissions

allowed) as timed tests. If learners take a test that has a hard due date or

retract date and no lates are allowed, they will see a timer. When the date

is up, the test will be auto-submitted, interrupting the learners if they

were in the middle of a question, but Mneme will save their last keystroke.

 

If a learner is in the middle of a test with a hard due date, and she saves

and exits, and then she returns to the test AFTER the due date has passed,

she will find the test auto-submitted (along with the answers).

 

Basically, we always save student answers - even if late submissions are not

allowed.

 

Redefined "Linear Order

---

In Mneme, linear order means that learners can answer (and not simply 'see')

a question only once. Learners can see a question, save and exit, go back to

it again, quite again, etc. They can get feedback (if enabled), remove

uploaded files (which is a round trip to another screen and back), and enter

/ exit as often as they like till they say they are done. Once done, they

cannot come back to the question.

 

If learners 'see' a question but don't answer it, we alert them to answer

it. But, we don't keep them stuck. They can proceed to Next, after the

alert, leaving a question empty.

 

If learners save and exit, we return them to the last unanswered question.

 

Special instructions are added at the beginning and top of each page of

linear tests to alert learners that they can only answer a question once,

etc.

 

Review page - Answer Key/Feedback

---

'Answer Key' and 'Feedback' are highlighted in bold.

 

Grader comments in the student review scores page are highlighted to stand

out. A note icon is added for additional emphasis.

 

List of Tests (to be done in 1.0.1)

---

Mneme will NOT have two lists of assessments - an active and an inactive

list. Instead, there will be one list of assessments that resembles the

Assignments Tool list. It will (probably) include the following information:

 

 

- Test title

- Open/due date

- Feedback date

- Number of attempts allowed/completed

- Status -- Not started, In Progress (save and exit), Graded, Ungraded,

Partly Graded (for essays)

- Time left (for timed tests)

- Score

 

Other fixes/enhancements

---

• Part titles of "Default" were visible to students despite spec/settings

(fixed).

• Removed statistics link; statistics will be part of the review page,

eventually.

• Removed file upload action buttons from student feedback/score page.
• Increased limit for short essay questions from 4K to 65K.
• Moved "Final Page URL" from the list of items to the final submission

comments in confirmation page.

 

 

 

 

--

Vivie Sinou

Dean, Learning Technology

ETUDES Consortium Project

Foothill College

 

---

 

This automatic notification message digest was sent to users of ETUDES (http://etudes-ng.fhda.edu)

You can modify your notification preferences in your My Workspace.