My script to run roms and save sram on linux

[Damien]

Hey there, I had trouble with the Retrode and Linux, especially with how the sram is saved back on the cartridge.  Snes9x seems pretty inconsistent, sometimes successfully saving on the Retrode, sometimes completely corrupting the sram. So in order to keep that from happening, I devised a script that would help me keeping my sram safe on both the Retrode and the hdd.

I'm using Ubuntu 11.04 and tested the saves with Chrono Trigger japanese version.
The emulator is snes9x-gtk 1.53 from https://launchpad.net/~bearoso/+archive/ppa

Code: [Select]

#!/bin/sh

set -v
# shows executed command

cp -r /media/RETRODE/ ~/retrode/media
# copies everything from the Retrode to the HDD.
# you need to have created ~/retrode/media/RETRODE first.

snes9x-gtk ~/retrode/media/RETRODE/*.sfc
# starts emulation of whatever is in the folder

wait
# waits until the emulator finishes

find /media/RETRODE/ -name '*.srm' -exec dd if=~/retrode{} of={} \;
# copies the save file .srm back to the Retrode

mv ~/retrode/media/RETRODE/*.srm ~/retrode/backup/
# moves the save file to a backup folder. overwrites the previous save.

rm -vf ~/retrode/media/RETRODE/*.*
# empties the retrode folder on your hdd to keep it clean.

Loads of thoughts:

I *think* I had some problem with the dd command: the first time I ran the script the sram was not saved (dd was successful, but reloading the game shown an unchanged savegame) and I had to put a umount /media/RETRODE at the end of the script. The problem is that you need to get up and hit reset on the Retrode before launching a game. But I tried the script again with the umount command and this time it worked well.  I'm not sur if I hallucinated or if there's a delay with dd or the OS before actually writing the data.

I tried the cat command but couldn't make it work, I don't know why.

I had loads of trouble using find -exec {} and couldn't just end up with the filename without the whole path, this is why the RETRODE mount path is replicated in the /retrode directory on the HDD. I'm guessing it has to do with -print0 or -printf "%f\n" pipe something, but I just couldn't figure it out. I'm a complete newb at that.

I'm quite unsure of the backup process though. With this script you keep a copy of the last sram in /backup so if for some reason writing on the cartridge failed then you still have a safe copy of it. Hovewer, in order to acknowledge the success or failure of the sram writing you need to launch the game/script again, which overwrites the sram file on the HDD with the one that is on the cartridge (the old one if failed, if you follow me). The only way currently to be safe is if you notice when launching your game that the previous sram writing failed: don't close the emulator, go to the /backup folder and save your .srm to a third location so it won't be overwritten.

I had no problem with Gens but I think I will make another similar script for softpatching: Gens wants the .ips patch to be in the same folder as the rom, which is not possible with the Retrode obviously.

[Matthias_H]

Nice! Here are some more snippets that may or may not be of use on your journey:

First line of a good shell script

Code: [Select]

#!/bin/bash

Somewhat smarter filename handling, check if file exists:

Code: [Select]

RDIR=/media/Retrode
TDIR=/var/tmp/retrode
mkdir -p $TDIR

ROMNAME=`ls $RDIR/*.sfc`

#Check if ROM exists
if [ -n "$ROMNAME" ]; then

# Strip path and extension
BASENAME=`basename $ROMNAME .sfc`

cp $RDIR/$BASENAME.sfc $TDIR

# Check if SRM exists as well:
if [ -f "$RDIR/$BASENAME.srm" ]; then
cp $RDIR/$BASENAME.srm $TDIR
fi

snes9x-gtk $TDIR/$BASENAME.sfc

fi

File comparison (content / date) etc.:

Code: [Select]

DIFF=`diff file1.ext file2.ext`
if [ -z "$DIFF" ] then
echo "Files are equal."
else
echo "Files are not equal."
fi

if [ file1 -nt file2 ] then
echo "File1 is newer than File2."
else
echo "File1 is not newer than File2."
fi

if [ -N file1 ] then
echo "File1 was modified since last read."
fi


[Damien]

Wow thank you, this is very useful. I'm writing my script again now.

With this script I noticed something with dd:

only the first dd is successful. So if you do dd1 / reset / dd2 / reset / dd3 / reset everything is fine and the save are incremented. But if you do reset / dd1 / reset / dd2 / dd3 / dd4 / reset / dd5 / dd6 / reset, only save1, 2 and 5 are written on the cartridge, no matter what dd or diff says.

This is the script I used, with the modification from Matthias:

Code: [Select]

#!/bin/bash

#path setup
RDIR=/media/RETRODE
TDIR=/var/tmp/retrode
IPSDIR=/home/damien/jeux/ips
BACKUPDIR=/home/damien/jeux/backup
mkdir -p $TDIR

# find ROM basename
ROMNAME=`ls $RDIR/*.sfc`
BASENAME=`basename $ROMNAME .sfc`

# copy the rom, sram and IPS in the temp folder
cp $RDIR/$BASENAME.sfc $TDIR
cp $RDIR/$BASENAME.srm $TDIR
cp $IPSDIR/$BASENAME.ips $TDIR

# start the emulator and wait
snes9x-gtk $TDIR/$BASENAME.sfc
wait

# copy the save file .srm back to the Retrode, to the backup dir and delete the temp dir.
dd if=$TDIR/$BASENAME.srm of=$RDIR/$BASENAME.srm
cp $TDIR/$BASENAME.srm $BACKUPDIR/$BASENAME.srm
rm -vrf $TDIR

#comparaison between the backup srm and the cartridge srm
DIFF=`diff $BACKUPDIR/$BASENAME.srm $RDIR/$BASENAME.srm`
if [ $BACKUPDIR/$BASENAME.srm -nt $RDIR/$BASENAME.srm ]; then
echo $BACKUPDIR/$BASENAME.srm is newer than $RDIR/$BASENAME.srm
else
echo $BACKUPDIR/$BASENAME.srm is not newer than $RDIR/$BASENAME.srm
fi

if [ -N $RDIR/$BASENAME.srm ]; then
echo $RDIR/$BASENAME.srm was modified since last read.
else
echo $RDIR/$BASENAME.srm was NOT modified since last read.
fi

if [ -z "$DIFF" ]; then
echo "Files are equal."
else
echo "Files are not equal."
fi

Maybe the problem is with the script, I couldn't say.

Please try to reproduce the dd failure, I know it's not easy to spot as you need to keep track of several different saves. It should behave the same with RETRODE.CFG .



So I believe unmounting is essential to keep saving the srm. Now I know how to unmount the Retrode in bash, but how do you mount your device back?


[edit] Come to think of it, the saving inconsistency that I noticed with Snes9x (which pushed me to make this script) may be because of the same bug. Maybe It is a problem within the retrode?

[Matthias_H]

It is a general problem that has to do with with the different possible ways of writing a file to a medium. Sadly, hardware constraints make it impossible to fix on the Retrode's end, but it should not be too hard to work around it on OS and application level.

Instead of unmount, have you tried the following command? It flushes the I/O buffers while leaving the device mounted.

blockdev --flushbufs [devicename]

[Damien]

I will try that. I will sound stupid but how do I find the retrode internal name? I had the same problem when trying to implement a unmount / mount cycle.

[Muzer]

You could also just try sync. Might be a bit slow if you have a slow hard drive, but it simply syncs data to *all* connected drives.



To find the device name:

type mount on its own

it will show a list of mounted devices and their device names. You'll obviously want to look for the device name for /media/RETRODE. It would be quite simple to do this programmatically:

mount | grep /media/RETRODE | cut -d " " -f 1

Not foolproof, but good enough for almost all situations. Obviously replace /media/RETRODE with whichever variable you want if you want to keep it flexible ($RDIR in Matthias_H's example I think)

Of course, if you want to get the output of that command and save it into a variable:

RDEV="$(mount | grep /media/RETRODE | cut -d " " -f 1)"

Or if you want to maintain compatibility with non-bash shells:

RDEV="`mount | grep /media/RETRODE | cut -d " " -f 1`"

[Damien]

Thank you.

sync did not work.
blockdev --flushbufs /dev/sdd (had to do it with sudo manually after the script as I had permissions problems) did not work.
umount /media/RETRODE did not work.

There was no error messages so I suppose each command functionned properly, but to our purpose it didn't solve our problem.

I really don't know what to do.
Currently only a RESET after each dd allows the following dd to be successful.
Currently only a RESET after each SRAM writing allows the following SRAM writing to be successful.


[edit] I tried cat instead of dd. Same behaviour.

I noticed something also when copying the rom. after a reset the rom takes a while to be copied and launched (Chrono Trigger is 4MB) but every subsequent launch of the script is quite fast. I would say it's another hint that something is happening with the buffer but I don't know how to explain it.

[Matthias_H]

It's called caching. The first time a file is read, the OS stores it in RAM and replaces future read accesses with cache lookups. This works under the assumption that the USB device won't change the file on its own (which the whole idea of "mounting" a drive is supposed to force). Same thing on the writing side: write accesses are accumulated, and only when a device is unmounted, everything is flushed to the drive. At least that's how I understand it should work. I am, in fact, having trouble understanding why "dd1; dd2; reset" should write dd1 and not dd2. If blockdev/umount/etc. have no effect, then the secret may be in the automounting mechanism of your particular distribution and/or desktop system.

[Damien]

It's a pretty straightforward Ubuntu natty 11.04 with long term support. I wouldn't say I critically modified it, if at all.

Are you able to reproduce this behaviour?

[Muzer]

It's incredibly odd, I find, that neither syncing nor unmounting is working. It really, really should.

[Damien]

Ok this is interesting.

I tried to test if RETRODE.CFG was affected as well, with nano.
When I first open, modify and save RETRODE.CFG, everything was fine.
When I reopen the file (without resetting) I can see my modifications.
However, if I try to modify and save it again, it first says:
"File was modified since you opened it, continue saving ?" Yes.
Then nano gives me an error:
"Error writing /media/RETRODE/RETRODE.CFG :  Input/output error."
This bug is repeatable.

So I can save once, but not twice. Clueless.

tho I found a bug report for nano about that https://bugzilla.redhat.com/show_bug.cgi?id=657875 . The guy was able to save its document nonetheless.

[edit] you don't even have to open it twice or to modify it.

- reset
- nano /media/RETRODE/RETRODE.CFG
- Ctrl-O, Enter (fine)
- Ctrl-O, Enter (bug)
"File was modified since you opened it, continue saving ?" Yes
"Error writing /media/RETRODE/RETRODE.CFG :  Input/output error."

[Matthias_H]

It's incredibly odd, I find, that neither syncing nor unmounting is working. It really, really should.

Unless the application secretly writes the data to a new file of the same name... Old problem, no good solution I'm aware of.

[Damien]

Unless the application secretly writes the data to a new file of the same name... Old problem, no good solution I'm aware of.

But then I wouldn't be able to save even once?

[Matthias_H]

I

Unless the application secretly writes the data to a new file of the same name... Old problem, no good solution I'm aware of.

But then I wouldn't be able to save even once?

I guess so. I'm afraid I don't have a Linux machine here for deeper testing. Anyone?

[Muzer]

I'll give it a go next time I'm at my Linux box.