Hacking WBFS Game Corruption Upon Deletion

[noobwarrior7]

so, I keep hearing about this bug, where, you delete a game from WBFS and "another" or "others" become "corrupt"

can we be a little more definitive please?

key questions, as I don't think I've experienced this...

Q1. To what degree are they "corrupt"? Code Dump? Game actually falters during play? Explain.

Q2. Is it one other game that messes up? Is it the game before? The game added before AND the game added after? All games added after?

Q3. Does it happen for everyone? Anyone deleted some/much and not had it happen once that they can tell?

Q4. For those who experience it, does it happen every time you delete a game?

Q5. Does size matter (hehe)? Aka: Hearsay about bigger game deleted than smaller game or some mumbo jumbo?

Q6. How can we even know if our games are corrupted by this bug?

thanks for your NEAT, LOGICAL, and english responses.

 

[AlexDP]

I'm not here to answer any of the questions you posted, but if your wondering why it happens, it seems to be a bug in the libwbfs code, which is used by all WBFS applications as far as I know, and as such all those programs will have the same bug. In fact, I'm pretty sure the same thing will happen if you do it with USB Loader on the Wii.

 

[noobwarrior7]

but thats the problem, I have no idea what the "same thing" youre talking about is....its unclear and there is literally TONS of contradictory hearsay, so if several people will answer in this format we'll have something that actually makes some sense.

and PS, your app rocks

 

[Renaldo225]

My understanding is that when you add a game it puts them physically on the drive in sequential order, with the new game starting just after the old game ends. (This scenario makes sense.) To add a little more detail, apparently a game is looking to be copied to the first open space on the disk, which is usually immediately following the last game installed.

Uninstalling changes that last sentence, because now you have , so the blank space will try to be used. Unfortunately WBFS doesn't seem to have any way to handle drive fragmentation or allow for file fragmentation of the ISO that gets copied over, so you get something like this:

ame 3>

I hope my little graphic makes sense. [-8 Basically my understanding is that the free space isn't checked to make sure there is enough, so if you copy a larger game into a spot that's too small it will overwrite the beginning of the next game. Result = code dump when you try to run the game after the one that was deleted (after meaning the one that was installed after, not after it alphabetically or anything else like that). If the game is smaller, then it will fit fine, but then the next game will be the one that overwrites, so the problem still exists, it's just delayed a little.

If I'm wrong on this, someone can tell you differently.

 

[AlexDP]

but thats the problem, I have no idea what the "same thing" youre talking about is....its unclear and there is literally TONS of contradictory hearsay, so if several people will answer in this format we'll have something that actually makes some sense.

and PS, your app rocks

Thanks

Yes, I agree, it's not entirely clear what happens when the game gets corrupted, and if people answer these questions it should be a bit more clear what happens and maybe even give some clues as to what exactly the bug is. We need steps to reproduce this problem, so post away people

Also, to the developers out there need to start looking into libwbfs' delete functionality and debugging it. I'll be doing that sometime in the near future. It would be much appreciated if anyone found the bug to post it publicly.

 

[FenrirWolf]

Q1. To what degree are they "corrupt"? Code Dump? Game actually falters during play? Explain.

-Code dump.

Q2. Is it one other game that messes up? Is it the game before? The game added before AND the game added after? All games added after?

-The game that was installed after the game that was deleted. Games that were installed beforehand are fine.

Q3. Does it happen for everyone? Anyone deleted some/much and not had it happen once that they can tell?

-Well, yeah. It'll happen to anyone who deletes a game and then installs another one (assuming it's not the most recent game added).

Q4. For those who experience it, does it happen every time you delete a game?

-If you delete the most recent game you added to your HDD nothing will corrupt. If you delete the second-to-the-last game you installed and install another one, then the last game installed on your HDD will corrupt. If you delete the third-to-the-last game and install another one, then the second-to-the-last game will become corrupt (then deleting and reinstalling the second-to-the-last game in an attempt to fix it corrupts the latest game on the drive).

And so on and so forth.

Q5. Does size matter (hehe)? Aka: Hearsay about bigger game deleted than smaller game or some mumbo jumbo?

-Well, let's say you added SSBB then you added a few other games after that. If you delete SSBB that'll leave a hole bigger than 6GB, so it might take a few new installs before you corrupt the next one in line (so far as I am aware). Conversely, let's suppose you installed something small like Animal Crossing first and then you delete it and install SSBB in its place. That sucker might end up overwriting the next few games in line entirely.

Q6. How can we even know if our games are corrupted by this bug?

-If you launch it and it code dumps, there ya go.

 

[AlexDP]

My understanding is that when you add a game it puts them physically on the drive in sequential order, with the new game starting just after the old game ends. (This scenario makes sense.) To add a little more detail, apparently a game is looking to be copied to the first open space on the disk, which is usually immediately following the last game installed.

Uninstalling changes that last sentence, because now you have , so the blank space will try to be used. Unfortunately WBFS doesn't seem to have any way to handle drive fragmentation or allow for file fragmentation of the ISO that gets copied over, so you get something like this:

ame 3>

I hope my little graphic makes sense. [-8 Basically my understanding is that the free space isn't checked to make sure there is enough, so if you copy a larger game into a spot that's too small it will overwrite the beginning of the next game. Result = code dump when you try to run the game after the one that was deleted (after meaning the one that was installed after, not after it alphabetically or anything else like that). If the game is smaller, then it will fit fine, but then the next game will be the one that overwrites, so the problem still exists, it's just delayed a little.

If I'm wrong on this, someone can tell you differently.

That's some good intuiton there... I think you may be right actually, I dont remember seeing any code that checks how much space is available before copying a file over. Still, dont quote me on that... I have to look into it again.

EDIT: FenrirWolf's response seems to further confirm this...

 

[FenrirWolf]

A piccy to clarify:

 

[noobwarrior7]

so....with a small possibility of error....


no code dumps = our games are "correctly" stored on the drive in the state they "should" be.

and it does not affect anything if you delete the last game added.

 

[FenrirWolf]

Yeah. So long as you haven't deleted anything, that is. Then there might be a code dump waiting to happen in your future.

Also, the WBFS gui I use warns you if the drive is defragmented when you try adding a game, so you could use it to check:
http://www.megaupload.com/?d=KF7N6YIH

 

[Renaldo225]

A piccy to clarify:

Sure, make my graphic look bad. . . I see how it is. . . [-8

 

[FenrirWolf]

Don't blame me, someone else made that pic and I just stole it.

 

[nIxx]

So basicly we would need to check between two installed games if there is enough space and if not go to next check again and so on till we find enough space between two games or it will install as last game only problem would be that so we waste some space.
But better as nothing

 

[BlindDude]

If we have to take such special caution.. Shouldnt we be using small flash drives... like 8-16gb?
OR just get a HUGE drive and DONT delete anything EVER? Until its fixed? lol

 

[Renaldo225]

So basicly we would need to check between two installed games if there is enough space and if not go to next check again and so on till we find enough space between two games or it will install as last game only problem would be that so we waste some space.
But better as nothing

Right, some kind of check would avoid the problem, but waste a minimal amount of free space. The long term solutions both have problems.

1 would be to allow file fragmentation. I don't know how hard it is to implement, because dadgumitall I've never needed to make a filesystem (and probably couldn't even if I wanted to). My guess is that it's not something that's trivial. The other problem is that it would probably cause slowness in games if the drive fragmentation got out of control.

Another would be to put all the free space at the end of the drive. That, of course, would be time consuming, especially if you removed a game that was installed pretty early on.

I think the best long term solution would be to check on install for drive space, and have some type of drive space defrag utility built into the loader. Once again I don't know nearly enough about the low level details of filesystems to know how easy or hard that would be, but it seems like it would have to be supported in the filesystem to begin with. Maybe I'm wrong and it's pretty simple. That would be nice. [-8

That last option would allow for normal use without a problem, with the ability to run a "disk space recovery" type job once a week or month, or when you run low on space.

Can people tell me A) If my understanding of the problem is correct and B) How difficult it would be to get the first part (check for disk space before install) done. If they are really ambitious they could tell me C) How hard would it be to put all the free space at the end of the drive (ie, move all the data to the front).

 

[nIxx]

If we have to take such special caution.. Shouldnt we be using small flash drives... like 8-16gb?
OR just get a HUGE drive and DONT delete anything EVER? Until its fixed? lol

Possibly one reason for the SD loader

 

[BlindDude]

If we have to take such special caution.. Shouldnt we be using small flash drives... like 8-16gb?
OR just get a HUGE drive and DONT delete anything EVER? Until its fixed? lol

Possibly one reason for the SD loader

That would suck because TONS of people just went and bought a bunch of hdd's..

Man,, now that I look back,, I shouldve bought stock in WD, Maxtor, and the rest... lol

 

[AlexDP]

So basicly we would need to check between two installed games if there is enough space and if not go to next check again and so on till we find enough space between two games or it will install as last game only problem would be that so we waste some space.
But better as nothing

Right, some kind of check would avoid the problem, but waste a minimal amount of free space. The long term solutions both have problems.

1 would be to allow file fragmentation. I don't know how hard it is to implement, because dadgumitall I've never needed to make a filesystem (and probably couldn't even if I wanted to). My guess is that it's not something that's trivial. The other problem is that it would probably cause slowness in games if the drive fragmentation got out of control.

Another would be to put all the free space at the end of the drive. That, of course, would be time consuming, especially if you removed a game that was installed pretty early on.

I think the best long term solution would be to check on install for drive space, and have some type of drive space defrag utility built into the loader. Once again I don't know nearly enough about the low level details of filesystems to know how easy or hard that would be, but it seems like it would have to be supported in the filesystem to begin with. Maybe I'm wrong and it's pretty simple. That would be nice. [-8

That last option would allow for normal use without a problem, with the ability to run a "disk space recovery" type job once a week or month, or when you run low on space.

Can people tell me A) If my understanding of the problem is correct and B) How difficult it would be to get the first part (check for disk space before install) done. If they are really ambitious they could tell me C) How hard would it be to put all the free space at the end of the drive (ie, move all the data to the front).

A)Yes, B) It's pretty easy to check for disk space first. C)it's entirey possible, not really difficult but not trivial either. It'd take some time to actually do the moving though.
Btw, have any of the developers out there used the wbfs_trim method in libwbfs explicitly? The (short, nearly useless) comment for it seems to imply that it would indeed fix up the file system, but I haven't really thought about the code too much (too tired to just now).

 

[AlexDP]

I've created a Developer's Discussion here: http://wiinewz.com/forums/backup-loaders/9...nd-libwbfs.html
If any devs are interested, please head there to discuss the future of libwbfs and WBFS.

 

[AllWiidUp]

Adding here in case my topic rolls off the screen

http://gbatemp.net/index.php?showtopic=147969