Zarf's script is something I wish I'd tried much earlier. It had the added bonus of helping me learn python. As wonderful as the Inform IDE's "test" command is, it has limitations:
* no undo, which is sort of a problem if you get killed
* only one test run, with restarts being awkward
* can't check the content
I've been tinkering with this in my own games, but I think I've found a sub-script that's flexible enough to share with others. Eventually I'd like to do a whole post on using Zarf's framework from Windows, but for now, here's an idea of how to get things rolling if you have the framework up.
It's one of my favorite axes to grind, fixing the "You can't go that way" generic command. It's always meant restriction more than exploration to me, and changing it helps me develop my world, and I look for it in games I test. So here is my Python script:
https://github.com/andrewschultz/miscellany/blob/master/python/noway.py
This needs a text file in the same directory, labeled reg-noway.txt, to start. I ran this test on The Problems Compound, which had a test file at https://github.com/andrewschultz/the-problems-compound/blob/master/testing/reg-pc-noway.txt converted from https://github.com/andrewschultz/the-problems-compound/blob/master/testing/roomlist.txt.
You can see the general syntax--just list the name of the rooms, and adjust the binary names accordingly. Obviously, this can be more flexible as I build the script, but the main
thing is that it will take a text file full of rooms and spit out a
bunch of tests. I like this because insta-death movements are neutralized, and the "undo" means you can try a different direction right away.
I also disabled diagonal directions in some games, which means a lot fewer tests need to be run.
But this only does so much. You want a working walkthrough, with the right clues. So I spruced up https://github.com/andrewschultz/the-problems-compound/blob/master/testing/reg-pc-thru.txt and just made sure everything worked okay...and it turned out I had a few typos to look through.
Thus my fears of just doing-over work I already did, and not really getting anything, were misplaced. I even tuned up some double-text I should've seen before. I think just the process of sitting down and creating the test cases switches the focus from "I should've tested this, I think" to "I'm going to make sure I tested this." And if I cheated with some GOTO commands, well, I was able to cover the major walkthrough and check off on more than just "you won."
It's already helped me fix a few trivial annoying bugs, because I
could just slip in a test and say "OK, does this work? I'd hate for this
to break." And so I've gotten a lot of mileage. And I suspect it'll help you, too, but I'll write more later.
Still, the short take-away is: do it on a game you want to re-release. Or a WIP. It provides a lot of security that stuff you fixed will stay fixed. Or you'll know when it broke.
One of the benefits of using glulx-typescript is that we've already created a node/CLI version. This allows you to run commands from the command line and do whatever you want with the results (compare, report). And when checking results, you can focus on the portions of text that have changed, usually in the main text, but sometimes an unseen inventory, location name, score, or other items of interest that you may have "separated" from the main text.
ReplyDeleteEventually I will build tools to enable detailed testing scenarios, but for now the node/CLI version of glulx-typescript is perfectl capable of enabling deep testing.
Wow! That sounds handy and intriguing. I assume it's at https://github.com/thiloplanz/glulx-typescript?
DeleteI'm a bit worried my games might break it since I do weird stuff, but I'd like to give it a try. Let me know what sort of bug reports might be useful. I'd love to have another way to do automated testing so I leave my testers less on an island.