tag:blogger.com,1999:blog-2564777502681463717.post1113463586386734001..comments2023-12-31T10:17:57.560-05:00Comments on Teaching, Playing, and Programming: Program Testing For The Sake Of LearningChris Okasakihttp://www.blogger.com/profile/18247315355264748920noreply@blogger.comBlogger8125tag:blogger.com,1999:blog-2564777502681463717.post-24433656239670932362008-04-06T22:26:00.000-04:002008-04-06T22:26:00.000-04:00As a graduate of 3 of Dr. Okasaki's classes I can ...As a graduate of 3 of Dr. Okasaki's classes I can say that I've found the testing very useful. Something I don't believe he necessarily mentioned is that with the output from the tester readily at hand, students are far better informed when they come in for questions. This includes questions among their peers. Knowing the specific input that is crashing your program combined with the output allows the student to analyze their code and come up with possible reasons for bugs. If they aren't able to solve the problem on their own, they can at least speak intelligently about what the problem is, and are often more easily coached towards the correct solution. <BR/><BR/>However, as a student I also witnessed a downside to this approach, albeit it only applies to students who are less interested in their learning than their grade. A common effect of this approach is for peers to communicate "quick fixes" based on what test cases are being failed. EX Hey Tom its Bill, I can't pass test case 12. Bill you forgot such and such base case in your recursion, (or worse yet) Bill just copy this into your code.<BR/>While learning oriented students can use the test cases to coach each other, it is more often used to fix the problem without understanding the cause and learning to fix it on your own.Benhttps://www.blogger.com/profile/00779165409144943135noreply@blogger.comtag:blogger.com,1999:blog-2564777502681463717.post-90081884984315241702008-03-20T08:18:00.000-04:002008-03-20T08:18:00.000-04:00Chris,The feedback is immediate. The students see ...Chris,<BR/><BR/>The feedback is immediate. The students see their output (and the reference) only for examples. For tests they only get a score proportional to how many passed (and I tend to keep "1 point = 1 test"). Here's what a student says:<BR/><BR/><I>[...] i myself thought that i had a working program when i first submitted it only to have it pass 3 tests [out of 20], it took some trial and error before i was able to fix the errors and end up with a satisfactory grade. But it also helped me better understand how the program worked and what was being asked.</I>rgrighttps://www.blogger.com/profile/02991214367108471744noreply@blogger.comtag:blogger.com,1999:blog-2564777502681463717.post-15891656498428481522008-03-19T23:35:00.000-04:002008-03-19T23:35:00.000-04:00Chris,I recommend looking at Turing's Craft. They...Chris,<BR/><BR/>I recommend looking at <A HREF="http://www.turingscraft.com/" REL="nofollow">Turing's Craft</A>. They don't do exactly what you propose, but they may be close enough to be useful. I've never used their tool, but their demo looks pretty good.Chris Okasakihttps://www.blogger.com/profile/18247315355264748920noreply@blogger.comtag:blogger.com,1999:blog-2564777502681463717.post-3462687451498464872008-03-19T23:23:00.000-04:002008-03-19T23:23:00.000-04:00I'm not so familiar with TopCoder, and apart from ...I'm not so familiar with TopCoder, and apart from some half-hearted work with QuickCheck, I'm not sure I have the full-blown unit-testing religion either. But this seems like a very useful application.<BR/><BR/>I've been toying with an idea to provide a similar kind of <I>immediate, automatic feedback,</I> and just today I posted a <A HREF="http://contrapunctus.net/index.php/2008/automated-programming-tutor" REL="nofollow">mockup </A> of a tool (vaporware for the moment) to help beginners practice the rudiments of programming. (Flash+audio required to see my short presentation about the idea.) <BR/><BR/>I'd be thrilled to get some feedback on the idea from you, Chris, and the great community you're building here.Chris Leaguehttps://www.blogger.com/profile/14441920124568924031noreply@blogger.comtag:blogger.com,1999:blog-2564777502681463717.post-87408522020289660182008-03-19T21:35:00.000-04:002008-03-19T21:35:00.000-04:00Harald,You're absolutely right. The ability to lo...Harald,<BR/><BR/>You're absolutely right. The ability to look at other people's solutions is an important part of the TC practice rooms that I have not tried to replicate. And not just one person's solution, but several, written in different styles, perhaps in different languages. And not being quite sure that each solution is correct, needing to look at each one critically.<BR/><BR/>All this can be great for learning, but it's obviously awkward when grades are involved.Chris Okasakihttps://www.blogger.com/profile/18247315355264748920noreply@blogger.comtag:blogger.com,1999:blog-2564777502681463717.post-6775313366856593312008-03-19T18:40:00.000-04:002008-03-19T18:40:00.000-04:00When I was a student, I used to do that, too: Go t...When I was a student, I used to do that, too: Go to topcoder and practice in the practice rooms. I considered participating in the contests, but since they so rarely fitted in with my schedule, I just timed myself on some of the practice tasks instead.<BR/><BR/>But what was really fun was to go over other people's old solutions, and see if you could find errors in them that hadn't been spotted yet. I found a lot, and I think I learned a lot from it...<BR/><BR/>Perhaps it's too negatively competitive to reward students for breaking each other's input with corner cases, but it really is very enlightening. One learns to look more at what could go wrong in a program, instead of being blinded by what one <I>wants</I> it to do.Harald Korneliussenhttps://www.blogger.com/profile/02909854185625282505noreply@blogger.comtag:blogger.com,1999:blog-2564777502681463717.post-8577279146820301862008-03-19T18:14:00.000-04:002008-03-19T18:14:00.000-04:00rgrig,That sounds pretty close in spirit. I assum...rgrig,<BR/><BR/>That sounds pretty close in spirit. I assume that when a student submits something, they get feedback immediately? What form does that feedback take for the hidden tests? That is, do they get to see the input and expected output for the hidden tests that they missed?<BR/><BR/>A server is probably a better solution than what I'm doing. Among other thins, a server lets you think about doing things like tracking the most common errors.<BR/><BR/>But I wanted to point out to people to don't have the one week, or the knowhow, to develop such a server, that they shouldn't let that stop them, because you can get by just fine with a little bit of testing code manually added to each assignment.Chris Okasakihttps://www.blogger.com/profile/18247315355264748920noreply@blogger.comtag:blogger.com,1999:blog-2564777502681463717.post-80112802624941381462008-03-19T18:03:00.000-04:002008-03-19T18:03:00.000-04:00This term I'm giving the lectures for an undergrad...This term I'm giving the lectures for an undergrad course. It took me one week to develop the server (no database though, just plain files) and overall I feel as that saved a lot of my time.<BR/><BR/>The server is used for much of the grading (70%). I believe that some of your concerns do not apply.<BR/><BR/><I>...testing when used for grading often breaks the tight feedback loop that is where my students learn the most.</I><BR/><BR/>Students get to submit as often as they like before the deadline with no penalty.<BR/><BR/><I>testing when used for grading often breaks the tight feedback loop that is where my students learn the most.</I><BR/><BR/>Just like in TC, examples are public and tests are hidden. The examples have the benefit that it's easier to see blunders. The (hidden) tests have the advantage that they make students think hard about the problem.<BR/><BR/>Even if the grading is automatic I still look at what students submit and provide (public) feedback. (I noticed that most of them have use Math.pow(2,k) instead of (1<<k) so I explained why each expression does; I noticed that some appear to use the server as a compiler so I explained how to compile locally and how to come up with your own tests. And so on.)<BR/><BR/>There are still problems:<BR/>1. Students ask few questions on the forums.<BR/>2. I believe that some got discouraged and don't even try to submit; all those that try do well.<BR/>3. I think I (still) have the tendency to pick problems that are too hard.rgrighttps://www.blogger.com/profile/02991214367108471744noreply@blogger.com