Time for a "breakup"?

[WhosAsking]

This is an honest request from a much-pleased cheat engine user.

As we have seen lately, the number of games MAME supports has increased substantially. So, inevitably, the size of our cheat files has increased in kind.

I'm beginning to think that a single cheat file may be becoming too large for reasonable management. I also think it's affecting load times, since the cheat engine has to comb through the entire file to get every cheat for just one game.

Do you think it would be a better idea to break up the cheats into separate files, one for each game? The cheats could be organized into a MAME subdir much as the CFG, HI, NVRAM, etc. are organized now. That way, MAME only has to hunt down say "gamename.cht" and simply load up all the cheats from that file, no seeking. I think it would be much faster to load and much easier to organize (and much easier to locate and trim out space-robbing duplicate cheats).

Now, I can see a drawback: when a bunch of cheats for a bunch of games are posted, as is the case for a good deal of the time here. But that could be solved with a simple parser which takes old-format cheats such as are posted here and appends them to existing files in the new format. This could also be used to convert an old CHEAT.DAT to the new system. I could code a PC cross-platform version myself, if necessary, but I'd need help on the Mac end.

In any event, I thought I'd bring up the subject and see if it's worth trying.

[Pugsy]

It is true it's an absolut swine to manage I admit, hence why I start and then stop all the time. However single dat files for single games is not the way forward. It's been mentioned before and here's some points:-

1. The seperate files would require a lot more disk space then the single cheat file. At the moment the cheat file is about 5 megs with cheats for over 3000 games. Worst case scenario, depending on the cluster size you could be looking at 32K for each file....You be looking at HD usage of about 94 Megs!. On a 4K cluster you'd be looking HD usage of 12 Megs minimum.

2. A lot of seperate files would make it a right pain to maintain, an example a new cheat is found for the game Space Invaders you have to have a clone list and then load every clone cheat file picking the filenames one by one and add the cheat and test it. At the moment it's almost a case of search and replace - which is a lot quicker with a single file in a decent text editor.

3. If a game changes it's shortname you've got to manually rename the file and then load the effected cheat file and then search and replace the gamename. With the big cheat file it's just a straight search and replace.

4. On a decent spec PC you are looking at fractions of a second to parse a cheat file. If you make a single game cheat.dat and compare it with the 'big' cheat.dat file you shouldn't notice any great difference in load times.

5. Would make it even harder for 'lamers' to modify their cheats with cheats posted on the messageboard, I have enough emails now without trying to explain you have to find the file with the shortname of the game you want to add and then paste the cheats into that file. Then repeat the procedure for every game. For instance a new MAME is posted and Steph posts cheats for 20 new games they have to manually create 20 empty files with the short gamenames and then copy and paste the required cheats into the requisite file then save it in text format. At the moment you just copy and paste the full post into cheat.dat and resave in text format...and that causes enough problems!!!

To sum up....bad idea, would be a nightmare to maintain with the sheer number of games in MAME if nothing else.

[kelvSYC]

MESS does break things down by system, but then again, more games are supported in MESS compared to MAME.

[WhosAsking]

I see that the issue has been discussed before. This time, though, I'll counter your points.

1. The seperate files would require a lot more disk space then the single cheat file. At the moment the cheat file is about 5 megs with cheats for over 3000 games. Worst case scenario, depending on the cluster size you could be looking at 32K for each file....You be looking at HD usage of about 94 Megs!. On a 4K cluster you'd be looking HD usage of 12 Megs minimum.

True, but we're forced to use separate CFG and HI files already, and they're an even bigger waste of space than CHT files would be (they average only about 100 bytes each). As the average CHT would be bigger, they're less likely to waste space than CFG files would. Anyway, hard drive space is cheap nowadays. Unless one's system is old or very crowded, space is usually a non-issue.

And there's another way to alleviate the issue: ZIPLib, which is already in MAME to support the ROMs and samples. If the entire directory were compressed into a ZIP rather than stored as a directory, the cluster issue would be solved and you'd save space. The only reason I don't like the idea is because it makes updating cheats (especially manually) a pain.

2. A lot of seperate files would make it a right pain to maintain, an example a new cheat is found for the game Space Invaders you have to have a clone list and then load every clone cheat file picking the filenames one by one and add the cheat and test it. At the moment it's almost a case of search and replace - which is a lot quicker with a single file in a decent text editor.

Include support for both systems. Make it switchable. Best of both worlds.

You could also include parent cheat support much like parent ROM support. You'd just need a little flag for child cheat files to indicate that you wish to ignore the parent file (in case they don't match). Fortuantely, most clones match, so this would actually knock out a lot of the clone redundancy and actually reduce tedium by making one update work for all the children.

EDIT: Slight change to my own idea after realizing that automatic inheritance is a little tricky in the cheat engine. In a child cheat, one could add support for a simple command on the top of the file that says to include cheats from another file. With a little anti-repeat check, that solution becomes more workable.

3. If a game changes it's shortname you've got to manually rename the file and then load the effected cheat file and then search and replace the gamename. With the big cheat file it's just a straight search and replace.

Simply forgo the game name in the separate file system. Every cheat in the file would be for that game, so there's no need for the name within the file. If its shortname changes, it would simply be a matter of renaming one file, which is even easier than a search and replace.

4. On a decent spec PC you are looking at fractions of a second to parse a cheat file. If you make a single game cheat.dat and compare it with the 'big' cheat.dat file you shouldn't notice any great difference in load times.

Still, it would save time, especially for those with older systems.

5. Would make it even harder for 'lamers' to modify their cheats with cheats posted on the messageboard, I have enough emails now without trying to explain you have to find the file with the shortname of the game you want to add and then paste the cheats into that file. Then repeat the procedure for every game. For instance a new MAME is posted and Steph posts cheats for 20 new games they have to manually create 20 empty files with the short gamenames and then copy and paste the required cheats into the requisite file then save it in text format. At the moment you just copy and paste the full post into cheat.dat and resave in text format...and that causes enough problems!!!

Like I said before, I could make an old->new converter pretty easily that could take an old format (such as is posted here) and happily append the new cheats into the new files: a simple CLI program I could make work in practically any PC-based platform. It would also double as an old CHEAT.DAT to new system converter. So all the lamers would have to do is paste the new cheats into a file, then run the converter through it. I don't know if you can get much simpler than that. I could potentially make such a converter more sophisticated by having it detect duplicate cheats and allowing the option of overwriting older cheats with updated ones. It could be automatic (for the uninitiated), prompted (for the nitpicky) or disabled (for the paranoid).

Now do you still think it's a bad idea?

[Pugsy]

I see that the issue has been discussed before. This time, though, I'll counter your points.

Countered eh? You sound like Stuttering Juddah, thought the nick was a bit obvious but I'm giving you a chance just the same.... Who are you really?

True, but we're forced to use separate CFG and HI files already, and they're an even bigger waste of space than CHT files would be (they average only about 100 bytes each). As the average CHT would be bigger, they're less likely to waste space than CFG files would. Anyway, hard drive space is cheap nowadays. Unless one's system is old or very crowded, space is usually a non-issue.

The major difference here is that you only have CFG and HI files for games you've actually played - if you've never played it they don't get created. And you are not forced to use them, for starters if you cheat the HI files don't get created at all....problem solved! On a side note your point about the HI files is another point for the current cheat.dat, check out the hiscore.dat file which was introduced as individual files are recognised as a rotten idea!

And there's another way to alleviate the issue: ZIPLib, which is already in MAME to support the ROMs and samples. If the entire directory were compressed into a ZIP rather than stored as a directory, the cluster issue would be solved and you'd save space. The only reason I don't like the idea is because it makes updating cheats (especially manually) a pain.

Ah right so there goes your earlier point of "I also think it's affecting load times, since the cheat engine has to comb through the entire file to get every cheat for just one game." Do you honestly believe that using a zipped dir of 3000+ small text files and then parsing one of them is going to be faster? Believe me when you find you have to update 1000s of cheats for 100s of games...this will NOT be easier for the current or future maintainers...

Include support for both systems. Make it switchable. Best of both worlds.

You could also include parent cheat support much like parent ROM support. You'd just need a little flag for child cheat files to indicate that you wish to ignore the parent file (in case they don't match). Fortuantely, most clones match, so this would actually knock out a lot of the clone redundancy and actually reduce tedium by making one update work for all the children.

EDIT: Slight change to my own idea after realizing that automatic inheritance is a little tricky in the cheat engine. In a child cheat, one could add support for a simple command on the top of the file that says to include cheats from another file. With a little anti-repeat check, that solution becomes more workable.

The idea of making clones cheats has come up before and although it has it's good points unfortunately the cheat file or cheat engine is not set to do it that way. The fact is there is a finite number of games left to be added to MAME so to do that now would be more trouble than it's worth. And besides which at the moment although it's basic it is clean and functional and easy to understand. If it ain't broke don't fix it.

Simply forgo the game name in the separate file system. Every cheat in the file would be for that game, so there's no need for the name within the file. If its shortname changes, it would simply be a matter of renaming one file, which is even easier than a search and replace.

Yes that would be a way around it, but it's not easier than a search and replace...a file reanme is no quicker. And it's nice to have the gamename as a way of error checking. If the gamename.cht gets accidently renamed that's it - the cheats are all gone for tha game.

Still, it would save time, especially for those with older systems.

You can't pick and choose what arguements you use, you said earlier :- "Anyway, hard drive space is cheap nowadays. Unless one's system is old or very crowded, space is usually a non-issue." Mhz is a non-issue by your earlier logic, and your idea of zipping 3000+ files is going to make it slower not faster!!

Like I said before, I could make an old->new converter pretty easily that could take an old format (such as is posted here) and happily append the new cheats into the new files: a simple CLI program I could make work in practically any PC-based platform. It would also double as an old CHEAT.DAT to new system converter. So all the lamers would have to do is paste the new cheats into a file, then run the converter through it. I don't know if you can get much simpler than that. I could potentially make such a converter more sophisticated by having it detect duplicate cheats and allowing the option of overwriting older cheats with updated ones. It could be automatic (for the uninitiated), prompted (for the nitpicky) or disabled (for the paranoid).

I've written a CLI program to split the cheat file into seperate files based on short gamename already (I use it for sorting when things get out of hand). It's not difficult to use, but CLI programs are not easy for 'lamers' to use. At the moment they just have to copy the & paste the cheats into the cheat.dat file - it's not going to get any easier than that AND THAT'S A FACT!

Now do you still think it's a bad idea?

[/quote]

I never said I thought it was a bad idea I said it was a TERRIBLE idea and I still believe that it's a stinker. Everything you've suggested will just mean more work - there are no benefits which are worth the extra effort. It's a massive step backwards!

[kelvSYC]

Do you know how many people just like you have been asking for this?

This debate has been repeatedly put down many times. Let's address your concerns:

Split cheat files are a really bad idea, and it makes things unnecessarily complicated. Seperate cheat files would draw, in essence, parallels to rom collecting. Pretty soon the cheat file would be bundled together with the rom, and that is how you ruin the fundamental concept of emulation.

Also note that the only games that seem to have large amounts of cheats are fighters and some shooters and "game" games (sports, mahjong, puzzles). Some games have one line of cheats in them! Now, one line of cheats is certainly much less than the indivisible block of hard drive space. With larger hard drives come larger block sizes. Not too good for 3000+ individual files, is it? Thus, to make your idea reasonable, you would either have to find more cheats or reduce the number of games. Well, we only have one option...

One thing I applaud (to Ian, Paul, and the others) is the simplicity of the cheat engine. Firstly, they are text files - files where you can open in your regular run-of-the-mill text editor and edit to your heart's content. Making 3000-or-so individual cheat files is bad, but zipping them is worse, since now you have all your files in binaries, and Windows Compressed Folder Support aside, it would be simply cumbersome to extract one file, add maybe two lines to it, and rezip it. Believe me that the large majority of MAME users don't run MAME32, which is the only version of MAME that would even remotely come close from benefiting from this.

Besides, as you all know, loading text is WAY faster than loading binaries.

Further tests (mostly by Ian, I believe) have shown that the only significant delay on load times comes from loading large encrypted roms (ie. Neo, CPS2, etc). Just ask stephh, who tests the cheats out on his P133. A P133 isn't a serious MAMEing system, but yet cheats load quite quickly.

And now, ladies and gentlemen, the common pit-trap of cheat finders and cheat engine critics alike (Paul, this should be in a second, more technical, cheat FAQ): Just because a game is a clone does not mean that the cheats will be the same as the parent! Examples include Williams games and those "rig things up on different hardware" games. Making "cheat clones" will do very little, and will only benefit those games with extensive lists of cheats which, ROM cheats aside, are identical (eg. ddsom).

If you compare Final Burn cheat files (where cheat clones exists) with their MAME equivalents, you will see what I mean.

Now what would happen if you combined the notion of cheat clones with the notion of separate cheat files? You would have yourself a monster that people would stay away from. The main reason why I personally went into cheat finding is that the system is simple, flexible, and centralized. With a separate structure like those mentioned previously, neither I nor many other aspiring cheat finders will touch this with a ten-foot pole just because it is overly complicated and the effort would not be worth the final result. It would simply be too complicated to remember separate commands to do stuff like cheat-clones, includes, ignores, and the like. Binarizing the cheat file would be even worse, as dedicated programs on specific platforms would be needed to read those files. Being a Mac user who uses MacMAME with little programming experience, I'd simply be cut out of the loop.

Okay, other concerns:

1. Who are you to call "everyone" old? The cheat engine has undergone a major revision in v.60. The majority of cheat finders are still using pre-v.60 conventions to write their cheats. This would mean your "ideas" would quickly make this cheat engine obsolete.

2. "PC-based platform"? Are you stupid? I could insult you further, but I have some personal restraint. MAME was never intended to be specific to platforms, unlike most other emulators. Thus, such a program would leave many users (myself included) out of the loop.

3. You really need to learn the definition of a lamer. First, you are introducing an extra step into adding new cheats, by running a converter. Second, lamers ask for cheats that have either been:

- not found
- impossible to find
- posted recently
- believed not to work on their system (ie. I can't get this cheat to work...)
- for unemulated games
- for the wrong emulator

All in all, this is somewhat equivalent to "Do you want your website 100% (name of obscure and unsupported web technology like HotSauce) or 100% HTML?" Well, we have a clear cut winner here...

As I said before, the single strength of the MAME cheat file is its simplicity. Text files, copy and paste, those work for even a newbie to computers. Your ideas seem to unnecessarily complicate things. And if it's going to be the way the MAME cheat engine is going, I'll quit cheat finding right now.

[WhosAsking]

Okay, if you want to keep the old way, keep the old way. I never said we had to take the old way away.

I can see your point and the benefits of a single file, especially for the space-conscious and the cheat-hunters.

But for those of us who could afford to do so or those who wish to keep things a little more organized, I'd be willing to give it a spin. I wanted to add it on as an alternative method of organization that could be switched with the current method (via command line, CFG, or INI) if so desired.

Tell you what. I'll take a look at the cheat engine and see if I can code the necessary additions to the system to allow for the new system while at the same time keeping the old one intact. Heck, I might even be able to incorporate the cheat inheritance idea even the old system. This would cut down on those clone cheats that do repeat while not taking any ability away for those clones that don't repeat.

[stephh]

My comments about all this ...

First, I think that almost everybody is aware that I'm running MAME on the "crappiest" computers of the world : a P133 with 48 Mb of RAM, and even a 486SX-25 with 8 Mo of RAM ...

I admit that I haven't tested yet 'ddsom*' (the game with the biggest number of cheats !), but the number of cheats for a game CAN'T have an incidence on the loading time as all the text file(s) is (are) read from the begining to the end ...

FYI, I'm loading FOUR cheat databases for a total of a 8 Mb text file, and there is no MAJOR slowdown, neither on the loading time, and neither when you activate some or all the cheats !


Splitting the big cheat file into small cheat databases can be done (that's what I do to avoid merging WIP games that aren't in latest MAME release), but not the way you want it for multiple reasons (some have already been mentioned) :

1) A file for each single game would cause LOTS of loss space on the hard drive ! Imagine a DOS user with 32 Kb (or even 64 Kb !) clusters, all cheat files will need more than 100 (or 200) Mb ... And zipping the files in a single cheat.zip file would be worst, as this would mean, if you don't have zipmagic or zipfolder, to write code to support this feature and the IMPOSSIBILITY to save the cheats while you are in MAME !

2) Having single files would be harder to maintain, especially when the game gets renamed, and as Pugsy said, there is a risk that you lose a file, so you lose some cheats

3) A file for each single game would mean, "one file = one game (= one ROM)", so there would be ?#! questions such as "I have a xxx cheat file, so where can I get xxx ROM ?", and this is AGAINST MAME licence !

4) Even if I have do some tricks (= "removing" some drivers/games) to make CheckCDB work again, the tool is VERY useful and it wasn't meant to work with multiple files ...


The clones/parent relationship CAN'T be used for the following reasons :

1) This would mean to have an "equivalence" cheat list ... I know that Pugsy has written one (or sort of), but I don't want to bother with this ...

2) Most of the time, clones on different hardware than the "main" set use different addresses, which isn't surprising as the memory mapping is different ...

3) Some manufacturers (eg: Williams or Konami) seem to be completely rewrite the game code during their revisions, so the "parent" cheats are useless ...

4) Some clones don't have the same number of players than the "main" set, so you would have unneeded or missing cheats for them ...


IMO, if you want to split the main cheat database, you can try to create a file for each manufacturer, but the "best" idea would be to create a file for each driver/system (eg: one for NEOGEO games, one for CPS games, one for Taito F3 games, and so long) ...


I now have a question : why don't you want to use TEXT files ? You can't imagine how easy to work on the cheat database this way (especially for people who only want new cheats so they only have to copy/paste the ones from the board into their file) ! And even with this, there are still some people asking ?#! questions ...


I'm now wondering too why the cheat stuff should be for PC only while the best advantage of MAME (besides the number of games) is that you can run it on many platforms, and you'd be surprised to know how many *nix or Mac users there are ... I know that the previous cheat engine was mainly DOS-based, but that's why it has almost fallen into oblivion for about 2 years !


My final question is why would you want to change the cheat engine ? My girlfriend JCK has added many features in it for a year (mid 1998 to mid 1999), and it took about another year to Ian (who is a far better C coder than her or me) to rewrite everything and include the many submissions we have asked for during this time ! And I don't think that Nicola would accept to check thousands of code lines again coming from an unknown person ...


Feel free now to comment all this

Steph from The Ultimate Patchers

Visit The Ultimate Patchers' site

[ianpatt]

Okay, if you want to keep the old way, keep the old way. I never said we had to take the old way away.

I can see your point and the benefits of a single file, especially for the space-conscious and the cheat-hunters.

But for those of us who could afford to do so or those who wish to keep things a little more organized, I'd be willing to give it a spin. I wanted to add it on as an alternative method of organization that could be switched with the current method (via command line, CFG, or INI) if so desired.

Tell you what. I'll take a look at the cheat engine and see if I can code the necessary additions to the system to allow for the new system while at the same time keeping the old one intact. Heck, I might even be able to incorporate the cheat inheritance idea even the old system. This would cut down on those clone cheats that do repeat while not taking any ability away for those clones that don't repeat.

(deleted double post)

Executive Summary: I also disagree with this addition.

In my personal tests with a stopwatch, I found that the difference in loading time between loading aof3 with cheats enabled and with cheats disabled was small enough to be unmeasurable through simple observation. This test was done after underclocking the CPU down as far as the motherboard supports it. After five trials, averaging the two cases, the time taken to load the cheat database was 4/100ths of a second. This is below average human reaction time (do a google search.)

So, the time saved by using any other possible faster method is negligible.

Also, every once in a while, we get posts on this messageboard from newbies having trouble adding cheats to their cheat file. Making the system more complicated (especially if it involves running a program) will not help in this matter.

Having a compressed directory of individual files breaks the ability to write to the cheat database, which is unacceptable.

The parent/clone cheat concept is more complex than you acknowledge. Just because all currently found cheats are common among parents and clones, this does not mean that *all* the memory addresses are the same. Because of this, to feel "safe" creating a parent/clone relationship, we would have to manually compare the code for parent and clone, ensuring that there are no major differences. Only then could the cheats for a parent simply be copied to the clone. This would be very time-consuming (although it would be interesting to do... for a while) for anyone to do.

I see no advantage in adding this new storage method, therefore I ask you to not waste your time. The source code modifications are easy to do (I can think of a ~5 line addition that would accomplish this) but the long term effects of such a change are negative.

NOTE: I am not attacking you personally; so far you've handled this quite well. I only attack your idea.

[kranser]

Of course, if your cheat.dat file is too large, you probably have too many games in it.

Most people do not use every game in MAME, therefore by stripping out all the games you don't own/play, the cheat.dat file could be a lot more manageable.

Maybe we could have a program which renames cheat.dat, and strips out all games for which a CFG file doesn't exist - thereby, in theory, we'd only have cheats for games that we play.

Kranser.

[WhosAsking]

Sure, but some people out there would probably be willing to spend a little time streamlining cheats for a parent/clone system, finding games here and there for which it'd work and games that are best left in the old style. There'd be no real drawbacks once it was done (since all the cheats would load), it would be transparent to those who don't normally use it, and it would be relatively easy to update these kinds of cheats once done: fewer lines to repost.

PS. kranser, what about those people who have big CHEAT.DAT files because they actually do have a lot of games, or games for which the cheat lists are large (such as Mahjong games and some of the Neo-Geo games)?

[stephh]

I've finally loaded 'ddsom' and 'ddtod' on the P133, and I can confirm that loading the cheats has NO incidence !!!

Maybe we could have a program which renames cheat.dat, and strips out all games for which a CFG file doesn't exist - thereby, in theory, we'd only have cheats for games that we play.

A "better" idea would be to exclude from the CHEAT.DAT file the games that you don't legally own Evil Bonze whistles ...

Steph from The Ultimate Patchers

Visit The Ultimate Patchers' site

[WhosAsking]

Hmm...I'll still keep the cheat separation idea in mind, though not for use in the engine. It can still help to sort out and eventually compile all the cheats together for use either in a master file or in a database.

[kelvSYC]

Translation: You want to be Pugsy's successor when he retires from cheat finding and cheat file maintenance?

[stephh]

Translation: You want to be Pugsy's successor when he retires from cheat finding and cheat file maintenance?

If Pugsy had to retire definitively, I think I would stop sending updates and/or infos/bugs reports to the "moderator" of cheat databases ... I know you may not like it, but the only reason why I keep in touch with Pugsy is that he was the only one who accepted to host our cheat database (it was 4 years ago or so) ... And as I know he likes to rip cheats here and there (besides the fact that he finds/tests LOTS of cheats), that's my contribution to try to help him in having a "valid" cheat.dat file ...

Feel free to post your HATE here ...

Steph from The Ultimate Patchers

Visit The Ultimate Patchers' site