Case Study of an Optional, Ungraded AI Forum in a Graduate Course

Most AI tools get introduced with a promise. This post is about what happened when one was simply made available. In one online graduate course, students were told about Nuubi, nothing was graded, and no one had to use it. Here is what they did, what we compared it to, and what the numbers do not show.

Course Background and Participants

The course was a first-year graduate-level online course for working professionals. It enrolled about 30 students, and 29 of them are included in the engagement analysis. They came from a range of technical and professional backgrounds, so the questions they had varied a lot. Using Nuubi was optional and ungraded. There were no incentives and no participation requirement. That matters, because it means any activity reflects whether busy students found it worth their time.

Nineteen of 29 Students Used Nuubi Without Being Required To

Nineteen of the 29 students used Nuubi, more than 65 percent, and together they produced 208 interactions: posts, replies, and messages. Nuubi was opened 741 times over the term. Use held up late in the term too: in the final month there were 152 participations and 326 views.

The Default Course Forum Received No Posts

The course also had a built-in discussion forum. While some students viewed it, it recorded no student posts or comments in any section we looked at, including this one.

Why the silence? It is a common pattern. Jakob Nielsen’s widely cited 90-9-1 rule of thumb describes a steep participation curve: in optional forums, about 1% of users post, 9% comment, and the rest read. At Stack Overflow’s scale that still produces plenty of content. In a course section of about 30 students, it predicts roughly one student-authored post across three sections. A quiet default forum is the expected outcome. Nuubi’s activity stands out against that pattern.

One survey respondent said that seeing others ask similar questions and get answers helped them learn without having to ask for help themselves.

Comparison With Earlier Sections of the Course

After removing about 23 expected participations per student from new required assignments, participation was about 33 per student, nearly four times the earlier 8.6. The raw average for the Nuubi section was 56. Average page views per student also rose from 863.7 in the earlier sections to 1,558 in the Nuubi section.

Grades moved in the same direction but only slightly. The average final grade was about 90 against 87 with withdrawals included, and about 97 against 96 without D, F, and withdrawal outcomes. Those gaps are small compared with how much grades varied from section to section, so we are not claiming Nuubi raised grades.

Discussion of Early Adoption and Help-Seeking

Two earlier sections offer limited context: when Nuubi was introduced in week 3 of 8, 6 of 13 students opened it, 2 were active, and it received 15 contributions; in an earlier version without an AI assistant or LMS connection, 12 of 30 opened it, 2 posted or replied, and it also received 15 contributions. These observations fit the possibility that students settle on where to ask for help early, but the sections took place at different times and the earlier version differed in two ways. The instructor also said they did not see activity notifications in the week-3 section, which may have affected use.

Students Who Answered the Survey Wanted Nuubi in Other Courses

Six of the 29 students answered the end-of-term survey, a response rate of 21 percent, and the respondents chose to answer. With that caveat: four of the six said they would be very disappointed if Nuubi were removed, and all six said they would like it in their other courses (four yes, two maybe). Half rated it 10 out of 10 on willingness to recommend, with an average of 8.7. Students said it made asking for help easier, especially for people who like to watch others ask first.

The instructor reported fewer repetitive one-on-one emails, and that questions posted late at night were often resolved by the next morning. The instructor reported this observation; it was not measured.

This Study Cannot Show That Nuubi Caused These Results

This was one section of one course, compared with earlier sections instead of a randomly assigned group, so we cannot say Nuubi caused any of the differences. We do not claim a grade effect. The pilot also ran on an earlier version of Nuubi, in which Cuubi gave a single generic first answer that was not yet based on the course materials. The instructor often followed up to confirm the answer or correct it. We did not log how often, so we treat that as an observation. These numbers offer a starting point for a larger study.

Conclusions on Optional AI Forum Use

Students used an AI-supported forum on their own when nothing made them, and they used it in a place where the standard forum went unused. That is enough to justify a larger, better-designed look, and not much more. Students also chose Nuubi over the standard LLMs they already had access to outside the course. They said it helped to have everything in one place inside the course shell, to get answers from peers and faculty, and to see different ways of solving the same problem.

If you are planning your own pilot, what would you most want to learn from it?