Register - Login
Views: 86639529
Main - Memberlist - Active users - Calendar - Wiki - IRC Chat - Online users
Ranks - Rules/FAQ - JCS - Stats - Latest Posts - Color Chart - Smilies
10-24-17 12:49:09 AM

Jul - Game Research/Hacking/Modding - Migrating Pokémon from Generation 2 to 3 New poll - New thread - New reply
Pages: 1 2 3 4 5Next newer thread | Next older thread
GuyPerfect
Catgirl
Level: 64


Posts: 883/1093
EXP: 2211411
For next: 2686

Since: 07-23-07


Since last post: 101 days
Last activity: 59 days

Posted on 12-16-13 04:38:36 PM Link | Quote
Oy caramba. Every time I try to post this thread I end up writing an article. Well, not this time! Instead, I have a proof-of-concept ready! It's not user-friendly enough to be ready for prime time, but it gets the job done and it's a great start for planning some better software.

Like I mentioned in another thread, I'm working on a technical experiment for migrating Pokémon from Generation 2 to Generation 3. There's some big philosophical discussion that can go on regarding this subject, but I'm gonna skip all of that. The whys aren't important. It's the whats I'm here to share with you today.

I've done extensive work reverse-engineering the save data for Pokémon games. You can see most of my findings here, here and here.
__________

Stat Growth

This is the big 600-pound Hippowdon in the room. Generation 2 uses the old stat exp. mechanic, while Generation 3 uses the new(er) effort point mechanic. Converting between them is not graceful: you can achieve maximum growth for all stats under the old system, but not the new system. Migration therefore may necessitate a reduction in stats, which I'm sure some people would get all frothy-mouthed over. Legend has it this is the reason Nintendo opted to remake the generation 1 and 2 games rather than provide an official migration solution.

Either way, it's a problem that needs to be solved. I'm working on the premise that stat exp./effort points are a reflection of... well, experience and effort. They are not a means to an end: that is, they're not a stepping stone to the final stat value. They indicate a Pokémon's growth in their stats, but not necessarily the final numerical stats themselves. To that end, I established a policy that makes migration easy: growth by stat should remain relative to all the other stats.

Due to how stat exp. works (the base stats of the defeated Pokémon are added to the stat exp. on defeat), it's impossible to train up just one stat like you can do with EVs. This affords a simple migration technique: simply scale down all of the stats until they fit in the required range. This is a four-step process:

Calculate the stats using the stat exp. formulas
Reverse-calculate the corresponding EVs using the EV formulas
If the maximum EV of all stats is greater than 252, scale them all down so that the max EV is 252
If the total EVs are greater than 510, scale them all down further so that the total is 510

I'm open to suggestions for this. One thing I don't want to do is allow the player to pick and choose which stats they want max'd and which ones they don't. That's what Super Training is for. This should be automated and treat all stats fairly.
__________

Randomness

Generation 2 has 16 bits of randomness per Pokémon (4 for each of 4 IVs), while Generation 3 has 62 (5 for each of 6 IVs, plus 32 "personality" bits). This means additional bits need to come from somewhere. The OT ID No. is also twice as large in Generation 3, which presents an interesting opportunity...

I'm constructing a 32-bit OT ID by using the old OT ID as the lower 16 bits (since that will display the same 5-digit number in the Summary screen in-game), and the old IVs as the upper 16 bits. The resulting 32-bit value I then use as the seed for a pseudorandom number generator that is used to generate other random things, the personality value among them. Doing it this way guarantees that the data from a Generation 2 Pokémon will always result in the same Generation 3 Pokémon.

The algorithm I decided to use, due to its appropriate context, is the one used to encrypt the Pokémon save data records in Generations 4 and 5. It looks something like this:

uint32_t regi;

uint16_t PokeDSrand() {
regi = regi * 0x41C64E6D + 0x6073;
return regi >> 16 & 0xFFFF;
}

Output from this algorithm, after seeding with the 32-bit OT ID, is used for the initial personality value as well as selection for appropriate values of auxiliary attributes.
__________

Gender

In Generation 2, the Pokémon's gender is determined by the Attack IV. In Generation 3, the lowest byte of the personality value is used. Both are checking against the same value for the Pokémon's species, which represents the ratio of males to females for the species.

For Generation 2, the check is whether the Attack IV is greater than the gender threshold. If it is, the Pokémon is male. For Generation 3, the check is whether the lowest personality byte is greater than or equal to the gender threshold instead. The thresholds are pretty much the same: the 4-bit value for Generation 2 is the upper 4 bits of the 8-bit Generation 3 value, and the lower 4 bits are all 1s.

Using the random bits generated for the initial personality value, the Pokémon's random gender can be checked. If it happens to be the wrong gender, that byte can be corrected by simply subtracting it from 255.
__________

Unown's Letter

In Generation 2, Unown's letter is determined by concatenating the center 2 bits of the IVs, in the order of (high) Attack, Defense, Speed, Special (low). That 8-bit value is divided by 10 to yield a result from 0 to 25, which corresponds with the letter.

In Generation 3, the lowest 2 bits of each of the 4 bytes of the personality value are used instead, retaining their order (the highest byte contains the highest 2 bits, etc.). This value is then divided by 28, and the remainder is the letter. 26 is ?, and 27 is !.

Randomness should come into play here. There are 9 values that can express any given letter, where A, B, C and D can be expressed with 10 values. Ideally, a random value across all valid values should be selected. That's the easy part.

The harder part is accounting for shininess. Shininess is explained below, but basically we need to account for this: bits 2-3 and 6-7 of the generated letter value, when XOR'd together, can produce any of the 4 combinations of 2 bits. This combination needs to be the same as the XOR result of bits 8-9 and 24-25 in the OT ID or the Pokémon will not be shiny.

As luck would have it, for all 28 permutations of Unown, there's at least one valid value for each of the 4 combinations of two bits. All Unown can be shiny no matter what the OT ID is. So when an Unown needs to be shiny, its letter value is simply selected from a list of all the valid values that match both the letter and are shiny. The algorithm I wrote for this suits all permutations of Unown, but in Generation 2, the only Unown that can be shiny are letters I and V (coincidentally enough, considering shininess comes from the IVs in Generation 2).
__________

Hidden Power

Hidden Power's type and power both come from the IVs in Generations 2 and 3. Type is a 4-bit value, and both generations use the same list. Power can range from 30 to 70.

In Generation 2, type is a 4-bit value using the lower 2 bits of the Attack IV as its upper 2 bits, and the lower 2 bits of the Defense IV as its lower 2 bits.

In Generation 3, type uses a 6-bit value using the least-significant bit of each IV, in the order of (high) Special Defense, Special Attack, Speed, Defense, Attack, HP (low). This value is then scaled to the range of 0...15 with the expression "value * 15 / 63". For this reason, the maximum value of 15 can only come from an initial value of 63, making Dark the rarest type for Hidden Power.

In Generation 2, power comes from the most significant bits of the IVs, in the order of (high) Attack, Defense, Speed, Special (low). This value is multiplied by 5, then added with 3 (if the Special IV is greater than 2), or the Special IV itself. The result is then divided by 2, which produces a number from 0 to 39. 31 is added to this to yield Hidden Power's power.

In Generation 3, the same algorithm used to determine Hidden Power's type is used, but it uses the second least-significant bit from the IVs (same order). It's scaled to 40 (value * 40 / 63), then has 30 added to it. Likewise with the type, only IV bits of all 1s can produce the maximum value of 70.

Retaining Hidden Power's attributes in Generation 3 necessitates modification of the IVs; there's no way around that. Since stat growth consistency between generations is all mangled up to begin with, IV accuracy isn't of utmost importance. What I ended up doing was building a list of valid matching values and selecting one at random, just like I did for Unown's letter.
__________

Shininess

One in every 8,192 Pokémon is shiny. It means nothing, but people still bend over backwards for it.

In Generation 2, the 16-bit IVs value is AND'd with 0x2FFF, and if the result is 0x2AAA, the Pokémon is shiny.

In Generation 3, the personality value is XOR'd with the OT ID value, and both halves of the result are XOR'd with each other. If the 16-bit result is less than 8 (meaning, bits 4-15 are all zeroes), the Pokémon is shiny.

Checking whether a Generation 3 Pokémon is shiny is easy: just XOR down to that 16-bit mask value and see if it's less than 8. If it's not, you can easily use that mask value to "correct" the personality value and make it shiny. Here's what you do:

AND the mask with 0xFFF8, since the lowest 3 bits don't matter
XOR the upper half of the personality value with the new mask value: personality ^ mask << 16
Grab a cup of hot chocolate and pat yourself on the back

The only bits in the upper half of the personality value that shouldn't be corrupted are bits 24 and 25, which are used for the letter if the Pokémon is Unown. However, since those bits were selected such that the result after the XOR would be zeroes, using this "correction" can't possibly modify those bits because the corresponding bits in the mask have to be zeroes by this point.

In the event the personality value just so happened to be shiny when it wasn't supposed to be, you can "correct" that was well, resulting in "de-shinification", by XORing the upper half of the personality value with 0xFCF8. This time, it's a mask value that won't touch the Unown letter bits.
__________

Ability, Egg

The Ability bit (in the IVs field) is initialized as the least-significant bit of the final Personality value. If the species can only have one Ability (like our friend Unown), the bit must be cleared in the IVs field.

If the Pokémon is in an egg (denoted by a "sprite ID" of 0xFD in Generation 2), the Egg bit in the IVs field should be set, and the nickname should be changed to 0x60 0x6F 0x8B (タマゴ).
__________

Held Item

If the Pokémon is holding an item, it may be able to be migrated as well. Most items in Generation 2 also exist in Generation 3. Some items don't exist per sé, but do have a functional equivalent. Other items have no equivalent at all.

Incidentally, all TMs that teach a move in Generation 2 that can be taught by TM in Generation 3 share the same numbers as the Generation 3 TMs. Handy!

The following items translate into the same item:

Amulet Coin, Antidote, Awakening, Berry Juice, Big Mushroom, Big Pearl, Black Belt, Black Glasses, Bright Powder, Burn Heal, Calcium, Carbos, Charcoal, Cleanse Tag, Dire Hit, Dragon Fang, Dragon Scale, Elixir, Energy Powder, Energy Root, Escape Rope, Ether, Everstone, Exp. Share, Fire Stone, Focus Band, Fresh Water, Full Heal, Full Restore, Great Ball, Guard Spec., Hard Stone, Heal Powder, HP Up, Hyper Potion, Ice Heal, Iron, King's Rock, Leaf Stone, Leftovers, Lemonade, Light Ball, Lucky Egg, Lucky Punch, Magnet, Master Ball, Max Elixir, Max Ether, Max Potion, Max Repel, Max Revive, Metal Coat, Metal Powder, Miracle Seed, Moomoo Milk, Moon Stone, Mystic Water, Never-Melt Ice, Nugget, Paralyze Heal, Pearl, Poison Barb, Poké Ball, Poké Doll, Potion, PP Up, Protein, Quick Claw, Rare Candy, Repel, Revival Herb, Revive, Sacred Ash, Scope Lens, Sharp Beak, Silver Powder, Smoke Ball, Soda Pop, Soft Sand, Spell Tag, Star Piece, Stardust, Stick, Sun Stone, Super Potion, Super Repel, Thick Club, Thunder Stone, Tiny Mushroom, TM05 Roar, TM06 Toxic, TM10 Hidden Power, TM11 Sunny Day, TM14 Blizzard, TM15 Hyper Beam, TM17 Protect, TM18 Rain Dance, TM19 Giga Drain, TM21 Frustration, TM22 Solar Beam, TM23 Iron Tail, TM25 Thunder, TM26 Earthquake, TM27 Return, TM28 Dig, TM29 Psychic, TM20 Shadow Ball, TM32 Double Team, TM36 Sludge Bomb, TM37 Sandstorm, TM38 Fire Blast, TM44 Rest, TM45 Attract, TM46 Thief, TM47 Steel Wing, Twisted Spoon, Ultra Ball, Up-Grade, Water Stone, X Accuracy, X Attack, X Defend, X Special, X Speed

The following items translate into identical items with different names:

Berry → Oran Berry
Bitter Berry → Persim Berry
Burnt Berry → Aspear Berry
Gold Berry → Sitrus Berry
Ice Berry → Rawst Berry
Mint Berry → Chesto Berry
Miracle Berry → Lum Berry
Pink Bow → Silk Scarf
PRZ Cure Berry → Cheri Berry
PSN Cure Berry → Pecha Berry

The following items translate into different items with similar effects:

Mystery Berry (+5 PP) → Leppa Berry (+10 PP)
Polkadot Bow (+12.5% Normal Damage) → Silk Scarf (+10% Normal Damage)

The following items have no analog in Generation 3 and must be returned to the bag or PC (or deleted):

Berserk Gene, Black Apricorn, Blue Apricorn, Blue Sky Mail, Brick Piece, Eon Mail, Fast Ball, Flower Mail, Friend Ball, Gold Leaf, Gorgeous Box, Green Apricorn, Heavy Ball, Level Ball, Light Blue Mail, Love Ball, Lovely Mail, Lure Ball, Mirage Mail, Moon Ball, Morph Mail, Music Mail, Normal Box, Park Ball, Pink Apricorn, Portrait Mail, Rage Candy Bar, Red Apricorn, Silver Leaf, Slowpoke Tail, Surf Mail, TM01 Dynamic Punch, TM02 Headbutt, TM03 Curse, TM04 Rollout, TM07 Zap Cannon, TM08 Rock Smash, TM09 Psych Up, TM12 Sweet Scent, TM13 Snore, TM16 Icy Wind, TM20 Endure, TM24 Dragon Breath, TM31 Mud Slap, TM33 Ice Punch, TM34 Swagger, TM35 Sleep Talk, TM39 Swift, TM40 Defense Curl, TM41 Thunder Punch, TM42 Dream Eater, TM43 Detect, TM48 Fire Punch, TM49 Fury Cutter, TM50 Nightmare, White Apricorn, Yellow Apricorn

The following items cannot be held by a Pokémon:

Basement Key, Bicycle, Blue Card, Card Key, Clear Bell, Coin Case, Egg Ticket, Good Rod, GS Ball, HM01 Cut, HM02 Fly, HM03 Surf, HM04 Strength, HM05 Flash, HM06 Whirlpool, HM07 Waterfall, Item Finder, Lost Item, Machine Part, Mystery Egg, Old Rod, Pass, Rainbow Wing, Red Scale, S.S. Ticket, Secret Potion, Silver Wing, Squirt Bottle, Super Rod

A handy conversion chart is as follows:
    -0   -1   -2   -3   -4   -5   -6   -7   -8   -9   -A   -B   -C   -D   -E   -F

0- 0000 0001 0002 00B3 0003 0004 -- -- 005E 000E 000F 0010 0011 0012 0013 0014
1- 0015 0016 000D 0055 0056 0025 005F 0060 0061 -- 003F 0040 0041 0042 00DE 0043
2- 0044 004E 0062 00DF 006E 0050 0017 0018 0019 0049 0053 0054 004A -- 001A 001B
3- 001C 004B -- 004C 004D 004F -- -- -- 00B6 -- -- ++ -- 0045 0022
4- 0023 0024 -- -- -- -- -- -- 001D 00B7 0087 ++ 00CB 00D2 0085 0089
5- 0088 00D3 00BB 008C 0086 ++ 0067 0068 00BC ++ -- 00BD ++ ++ 00BE 00D1
6- 00D6 ++ 00CF ++ -- ++ 00CE ++ 00D9 00E1 00C2 00D4 00D0 008D 006A 006B
7- 00C3 00D5 ++ -- -- 00CD 00E0 00C4 -- 001E 001F 0020 0021 00CC 00C5 --
8- -- -- -- 006C 006D -- -- -- -- -- 00D7 002C 00C6 -- -- 00C7
9- 00D8 -- 00C8 -- -- -- 008A 00C9 ++ -- -- -- 002D ++ ++ ++
A- ++ ++ -- 00CA ++ ++ ++ ++ ++ 005D 00D9 -- 00DA 008B 008E --
B- -- ++ -- -- ++ ++ ++ ++ ++ ++ ++ ++ ++ ++ -- ++
C- ++ ++ ++ -- 0125 0126 ++ ++ ++ 012A 012B ++ ++ 012E 012F ++
D- 0131 0132 0133 ++ 0135 0136 0137 ++ 0139 013A 013B 013C -- 013D 013E ++
E- 0140 ++ ++ ++ 0144 0145 0146 ++ ++ ++ ++ ++ 014C 014D 014E 014F
F- ++ ++ ++ -- -- -- -- -- -- -- -- -- -- -- -- --
Value 0 represents no held item in both generations
Values marked "++" are items that can be held, but not migrated to Generation 3
Values marked "--" either cannot be held or are not valid items
__________

Other

Some other fields of interest:

Friendship, Pokérus and Move IDs are exactly the same in both generations (even for eggs)
Contest stats and ribbons should be initialized to all zeroes
The original location field should be set to 0xFE: Met in a trade
The original Poké Ball field should be set to 4: Poké Ball
The original game field should be set to 15: Not a main-series game

__________

Practical Example

After it was all said and done, I successfully migrated my original Ariados from Gold to FireRed using a custom utility. This was the play-through from when I first picked Ariados as my favorite.

The original:


The migrated:


During the testing phase, I did alter some attributes like gender, shininess and Unown's letter to make sure everything was working right. And it is. I haven't verified the Hidden Power stuff yet, but egads this is satisfying!

Take note of how the stats wound up. Skitters's total Stat Exp. exceeded 510 effort points' worth of growth, so things had to be scaled back a bit. The reason Special Attack and Special Defense are different is because of its Impish nature, which boosts Defense and nerfs Special Attack.

Now I'm really itching for an 8-bit cartridge interface on GBA so I can make this happen on the hardware.
Sanqui
1580
🦉
Level: 73


Posts: 1239/1581
EXP: 3364675
For next: 121193

Since: 12-20-09
From: Czech Republic

Since last post: 1 day
Last activity: 1 day

Posted on 12-16-13 05:01:27 PM (last edited by Sanky at 12-16-13 05:03:16 PM) Link | Quote

I'm constructing a 32-bit OT ID by using the old OT ID as the lower 16 bits (since that will display the same 5-digit number in the Summary screen in-game), and the old IVs as the upper 16 bits.


The full OT should be consistent between all of the trainers' Pokémon.


Retaining Hidden Power's attributes in Generation 3 necessitates modification of the IVs; there's no way around that. Since stat growth consistency between generations is all mangled up to begin with, IV accuracy isn't of utmost importance. What I ended up doing was building a list of valid matching values and selecting one at random, just like I did for Unown's letter.

This is slightly confusing. I'd prefer IV accuracy over Hidden Power accuracy (I'd perhaps make an exception if the Pokémon knows HP already).

Lastly, natures should be neutral, I guess.

As for the rest,
GuyPerfect
Catgirl
Level: 64


Posts: 884/1093
EXP: 2211411
For next: 2686

Since: 07-23-07


Since last post: 101 days
Last activity: 59 days

Posted on 12-16-13 05:48:31 PM Link | Quote
Originally posted by Sanky
The full OT should be consistent between all of the trainers' Pokémon.

In one sense, it already is. The number displayed on the Summary screen remains the same.

As for the full 32 bits, I don't see a compelling reason for that. I can't think of any context where it's ever been the case that the original trainer of a traded Pokémon mattered, since it's an outsider no matter what. For two outsiders to have the same original trainer, unless there's something I'm not thinking of, is entirely meaningless. Unless you already happened to know that the upper 16 bits differed, there'd be no way you'd ever find out.

Originally posted by Sanky
This is slightly confusing. I'd prefer IV accuracy over Hidden Power accuracy (I'd perhaps make an exception if the Pokémon knows HP already).

Even under ideal circumstances, IV compatibility between generations is impossible. The mechanics are different, and the IVs themselves can't even match up (a maximum value of 15 in Generation 2 yields different stats than a maximum value of 31 in Generation 3). Growth rates also differ: exp.-based stat formulas are exponential, while EV-based formulas are multiplicative.

It's a case of trying to fit a square peg into a round hole. Accurate IV conversion simply can't be done, so I don't consider it a problem to use the least significant bit towards Hidden Power compatibility.

Originally posted by Sanky
Lastly, natures should be neutral, I guess.

I thought about that. A case can definitely be made that all Generation 2 Pokémon had neutral natures, but there's already a precedent for new mechanics changing old stats: the incorporation of Special → Special Attack/Special Defense changed performance of many Generation 1 Pokémon traded into Generation 2. It's natural progression for something similar to happen when incorporating Nature.

This also brings us back to the IV thing. You can't have the same stats in Generation 3 as you can in Generation 2. Even if you discount the variance in the formulas, there's still the matter of 65,536 stat exp. values yielding more distinct stat numbers than the 252 effort points can. Trying to shoehorn stats from Generation 2 into the Generation 3 engine isn't just a square peg, it's a fool's errand.
Vegito
Random nobody
Level: 4


Posts: 1/3
EXP: 192
For next: 87

Since: 01-22-14


Since last post: 3.0 years
Last activity: 3.0 years

Posted on 01-22-14 10:04:08 PM Link | Quote
Hey Um can you send me a message or something? I want to know what program you are using so I can do this same thing.
GuyPerfect
Catgirl
Level: 64


Posts: 903/1093
EXP: 2211411
For next: 2686

Since: 07-23-07


Since last post: 101 days
Last activity: 59 days

Posted on 01-22-14 11:05:21 PM Link | Quote
The software was entirely self-made.
Vegito
Random nobody
Level: 4


Posts: 2/3
EXP: 192
For next: 87

Since: 01-22-14


Since last post: 3.0 years
Last activity: 3.0 years

Posted on 01-23-14 05:10:56 PM Link | Quote
Okay, well is there a way you can give me this self made program so i can use it?
GuyPerfect
Catgirl
Level: 64


Posts: 904/1093
EXP: 2211411
For next: 2686

Since: 07-23-07


Since last post: 101 days
Last activity: 59 days

Posted on 01-24-14 02:53:36 PM (last edited by GuyPerfect at 01-24-14 03:00:49 PM) Link | Quote
It's a hackish abomination of experimental code and hard-coded case resolution. I almost feel ashamed for having caused it to exist, so it isn't in a state that I feel comfortable distributing it.

Having said that, I'd be more than happy to help out if someone wants to put together something a little more formal to automate the process.
Vegito
Random nobody
Level: 4


Posts: 3/3
EXP: 192
For next: 87

Since: 01-22-14


Since last post: 3.0 years
Last activity: 3.0 years

Posted on 01-27-14 09:10:09 PM Link | Quote
Originally posted by GuyPerfect
It's a hackish abomination of experimental code and hard-coded case resolution. I almost feel ashamed for having caused it to exist, so it isn't in a state that I feel comfortable distributing it.

Having said that, I'd be more than happy to help out if someone wants to put together something a little more formal to automate the process.

sure where do I start! lol
Darkdata
Ruins!? ♥
Level: 98


Posts: 2758/2889
EXP: 9527209
For next: 127144

Since: 07-04-07


Since last post: 72 days
Last activity: 9 days

Posted on 02-20-14 02:03:39 PM Link | Quote
Hey, Guyperfect: Have you considered doing something with VBA and Lua? One script takes Gold/Silver data and places it in storage, and another takes the data and writes it to FR/LG/RU/SA?

If I had more programing knowledge, I would like to try it myself.
GuyPerfect
Catgirl
Level: 64


Posts: 906/1093
EXP: 2211411
For next: 2686

Since: 07-23-07


Since last post: 101 days
Last activity: 59 days

Posted on 02-22-14 12:02:42 AM Link | Quote
Oh snap, I totally didn't see Vegito's post a month ago. O-:

I'll definitely look into this project some more. If nothing else I'll whip up a Java program to do the conversion. I was so focused on making a C program that would easily port over to a GBA project that I wasn't focusing on getting the work done so much as dreaming about what might one day be...

It'll be a week, though.
GuyPerfect
Catgirl
Level: 64


Posts: 907/1093
EXP: 2211411
For next: 2686

Since: 07-23-07


Since last post: 101 days
Last activity: 59 days

Posted on 02-22-14 11:16:27 AM Link | Quote
Originally posted by GuyPerfect
It'll be a week, though.

Scratch that. I'm postponing the project I would have been working on this week, since I don't want to rush it. Instead, I'll work on this one!

Who's interested in seeing how it goes together? I'd welcome input as well.
2Tie

Rumours of my death have been greatly exaggerated.
Level: 57


Posts: 399/920
EXP: 1476632
For next: 9296

Since: 02-22-10
From: The Codrex, Karda Nui

Since last post: 5 days
Last activity: 59 min.

Posted on 02-22-14 01:00:57 PM Link | Quote
I would be!
GuyPerfect
Catgirl
Level: 64


Posts: 908/1093
EXP: 2211411
For next: 2686

Since: 07-23-07


Since last post: 101 days
Last activity: 59 days

Posted on 02-22-14 07:00:56 PM Link | Quote
Splendid! Okay, here's what I'm thinking...

The utility will be Java for three significant reasons: 1) it's not limited to a particular architecture or operating system, 2) it can package up resource files very nicely and with compression, and 3) it has feature-rich GUI development libraries. The GUI thing is the big one. I could write stuff in C, but cross-platform GUI stuff is way too involved for a simple little project like this. The alternative would be to forego the GUI and stick with, like, old-fashioned command prompt things like "Please enter the index of the Pokémon you wish to transfer."

Besides, with the source code distributed as Java (in the .jar package), it will allow people to easily tweak things to suit their tastes. Java really is the way to go for this. So hey, if you haven't already, grab your JRE!

I'm still brainstorming all of this, but the component structure I have in mind is like this:

Save - Loads an emulator .sav file and writes new ones. The source generation/game is detected, and certain fields like player name/geneder and PC are made available to the application.
Pokemon - A data structure representing a single Pokémon. The generation is supplied, and decode/encode routines are implemented. For one-way conversion from Generation 2 to Generation 3, a method is present that results in a new Pokemon object.
Item - A utility class and data structure combo. Instances only hold information about the item type and quantity, but there's a conversion function and support for the bag and PC storage collections.

Other than that, it's basically a GUI program that presents source and destination PC boxes for Pokémon, with in-game-style inventory management graphics. In addition to a simple migration interface, before-and-after views will also be available so the user can see what-all gets changed during conversion, if anything.

So hey, is anyone able to host this? I suppose it could be some kind of source repository, but really all we need at this point in time is, like... a DropBox. I suppose I could set up a DropBox.

Any input or ideas would be more than welcome.
GuyPerfect
Catgirl
Level: 64


Posts: 910/1093
EXP: 2211411
For next: 2686

Since: 07-23-07


Since last post: 101 days
Last activity: 59 days

Posted on 02-23-14 11:30:00 AM (last edited by GuyPerfect at 04-03-14 09:22:10 PM) Link | Quote
I'm looking through my code and I've got a list of attributes that apply to generation 2 and 3 Pokémon records.

Common fields:
OT ID
OT Name
OT Gender
Species Index
Gender
Held Item Index
Move (1, 2, 3, 4) Index
Move (1, 2, 3, 4) Current PP
Move (1, 2, 3, 4) PP Ups
Level
Exp.
Friendship
Pokérus Strain
Pokérus Days Remaining
Status Effect
Status Turns Remaining
Met Location
Met Level
Is an Egg
Unown's Letter (if applicable)
Is Shiny
Hidden Power Power
Hidden Power Type

Generation 2 exclusive fields:
IV HP (4-bit)
IV Attack (4-bit)
IV Defense (4-bit)
IV Speed (4-bit)
IV Special
Exp. HP
Exp. Attack
Exp. Defense
Exp. Speed
Exp. Special
Met Time of Day

Generation 3 exclusive fields:
Met Game
IV HP (5-bit)
IV Attack (5-bit)
IV Defense (5-bit)
IV Speed (5-bit)
IV Sp. Atk
IV Sp. Def
OT Language
EV HP
EV Attack
EV Defense
EV Speed
EV Sp. Atk
EV Sp. Def
Contest Cool
Contest Beauty
Contest Cute
Contest Smart
Contest Tough
Contest Ribbons
Ability
Nature
Poké Ball
Marking

Migrating from generation 2 to 3 requires some modifications to the data, but that's what was documented in the original post of this thread.

Now would be a good time to revisit exactly how to convert stat experience to effort points. I've expressed my ideas in the first post, but since this is going to be what's out there for the world to use, I'd really appreciate any input you guys have.
2Tie

Rumours of my death have been greatly exaggerated.
Level: 57


Posts: 401/920
EXP: 1476632
For next: 9296

Since: 02-22-10
From: The Codrex, Karda Nui

Since last post: 5 days
Last activity: 59 min.

Posted on 02-23-14 02:41:18 PM Link | Quote
It doesn't matter that much to me personally how stat experience is converted, whatever you had brought up earlier sounded pretty good. My question would actually be more on how you'd handle the Met Game value, unless Gen 3 has a value for Johto
GuyPerfect
Catgirl
Level: 64


Posts: 911/1093
EXP: 2211411
For next: 2686

Since: 07-23-07


Since last post: 101 days
Last activity: 59 days

Posted on 02-26-14 09:43:02 AM Link | Quote
Generation 3 doesn't have location data for anywhere outside the Generation 3 games, with the exception of the generic "Met in a trade" option that's used for side-games like Pokémon XD. That's the only one that makes sense for migrating Generation 2 mons.

Anyhoo, hooray for the weekmid! It's my day off!

Okay, so... Stat growth algorithms. Every stat of every Pokémon species has a "base" value. These do not change between Generations 2 and 3.

In Generation 2, the IVs are 4 bits in size and stat Exp. ranges from 0 to 65,535. The HP stat is calculated accordingly:
stat = (iv + base + sqrt(exp) / 8 + 50) * level / 50 + 10
All other stats are calculated like this instead:
stat = (iv + base + sqrt(exp) / 8) * level / 50 + 5
I've noticed some discrepancies between typical implementations of these formulas and the ones actually used in Gold and Silver (and likely Red/Blue/Yellow). They work for the most part, but every now and again, my results are off by 1. I've investigated this to an extent, but the code is a mite complex given that the Game Boy has to do things like multi-word square roots. If anyone feels up to the challenge, the stat calculator function is located at 03:6181 in the US release of... of... crap, I can't remember if my notes were for Gold/Sliver or for Crystal.

The important thing to note here is that the stat Exp. is fed into a square root operation during calculation of the resulting stats. This means that stats grow more quickly early on, and the more Exp. a Pokémon has for a given stat, the slower it grows.

Given base stats of, say, 100, and a level of 100, here are some sample values:

310 = HP, IV 0, Exp. 0
373 = HP, IV 0, Exp. 65535
340 = HP, IV 15, Exp. 0
403 = HP, IV 15, Exp. 65535
205 = Other, IV 0, Exp. 0
268 = Other, IV 0, Exp. 65535
235 = Other, IV 15, Exp. 0
298 = Other, IV 15, Exp. 65535

In Generation 3, the IVs are 5 bits in size and stat effort points range from 0 to 252. The HP stat is calculated like this:
stat = (iv + base * 2 + ev / 4 + 100) * level / 100 + 10
And other stats like this:
stat = ((iv + base * 2 + ev / 4) * level / 100 + 5) * nature
The Pokémon's Nature may modify a stat by 10%. Hindering natures become a 0.9 multiplier, and boosting natures become a 1.1 multiplier.

Notice this time that the effort points (analogous to stat Exp.) are used directly rather than first through a square root. The implication is that a stat will continue to grow at the same rate from start to finish, meaning it's far easier to train a stat now than it was in Generation 2. Note also that it's divided by 4, making effort values of 253, 254 or 255 not provide any benefit over 252.

For base stats of 100, level 100 and a neutral nature, we get the following sample values:

310 = HP, IV 0, EV 0
373 = HP, IV 0, EV 252
341 = HP, IV 31, EV 0
404 = HP, IV 31, EV 252
205 = Other, IV 0, EV 0
268 = Other, IV 0, EV 252
236 = Other, IV 31, EV 0
299 = Other, IV 31, EV 252

Pretty nice, huh? There's a couple off-by-one results, but for all intents and purposes, the minimums and maximums in Generation 3 are identical to those in Generation 2 (Nature notwithstanding).

When migrating from Generation 2 to Generation 3, ideally the Pokémon's stats will match values in both generations. That in mind, we need a formula that works backwards from the stat value to determine the number of effort points required to express the stat in Generation 3. It just takes a little bit of algebra; nothing fancy.

For HP:
ev = ((stat - 10) * 100 / level - 100 - iv - base * 2) * 4
For other stats:
ev = ((stat / nature - 5) * 100 / level - iv - base * 2) * 4
These results should be rounded to yield the closest valid effort point value. Logically, anything that should happen to result in an EV greater than 252 should be clamped.

It'd be nice if we could call it a day with this, but there's one more wrench in the works: Generation 3 restricts Pokémon to having no more than 510 effort points total. For fully-trained stats, this means you can have 252 in two different stats, with 6 points to spare. In Generation 2, on the other hand, you can have 65,535 Exp. in all six stats to have them all be fully-trained.

Like I mentioned in the first post, some amount of reduction may need to occur, and it needs to be objective and fair. The best I could come up with is that if the total number of effort points after the migration algorithm is applied exceeds 510, then all stats should be reduced in value by the same factor until they can be represented with 510 effort points. Doing it this way retains stat values relative to one another such that stats trained more in Generation 2 will still be the same relative amount stronger than the other stats in Generation 3. And that makes sense to me because you can't pick-and-choose which stats to grow in Generation 2: stat Exp. always grows for all stats for each opponent defeated.

This gives us the following expression:
if (total_evs > 510) stat = (stat * 1020 + total_evs) / (total_evs * 2)
Regarding the IVs themselves, the conversion from 4-bit to 5-bit is fairly straightforward:
iv_g3 = (iv_g2 * 62 + 15) / 30
While this provides the closest possible IVs to retain stat values, they may need to be modified after the fact. The lowest two bits of the IVs determine Hidden Power's attributes, and in most cases, something's gonna have to be changed in order to retain Hidden Power compatibility. Ideally, the IVs should be changed as little as possible.

For each type and power for Hidden Power, there's something like 2-3 different combinations of IV bits to represent each value. Simple brute force can be used to select the type/power combination that alters the IVs the least.
__________

As an aside... The Hidden Power type and power values use a naïve linear transformation from [0, 63] to [0, 15] and [0, 40], respectively. The formula they use is this:
output = input * max_output / max_input
This has an implication in that only the maximum input value of 63 can result in the maximum output value. For type, this is especially nasty because it means only 1 / 64 Pokémon can have a Dark-type Hidden Power, where it should be closer to 1 / 15. A better formula uses the usual "round to nearest" approach, where you add 0.5 to a value prior to truncating it to an integer. You can do this with integers only using the following formula (which I used in my stat-reduction and IV conversion formulas up there):
output = (input * max_output * 2 + max_input) / (max_input * 2)
Using that formula on the Hidden Power type, each type could have been represented with 4 to 5 different values (or 3 at the ends), meaning Dark could have been present on 3 / 64 of Pokémon, which is pretty darn close to 1 / 15.
GuyPerfect
Catgirl
Level: 64


Posts: 912/1093
EXP: 2211411
For next: 2686

Since: 07-23-07


Since last post: 101 days
Last activity: 59 days

Posted on 02-26-14 09:07:35 PM Link | Quote
I was so caught up in being smart that I did a very dumb thing.

Okay, here's the deal. Stat Exp. ranges from 0 to 65,535, but the square root of that amount is used in the formulas, meaning 0 to 255. Stat EVs, on the other hand, range linearly from 0 to 255 (from a practical standpoint, 0 to 252). See where this is going?

I did some testing and, um, yeah, the numbers line up. Converting Stat Exp. to the corresponding EV is, well... it's this:
ev = sqrt(exp)
In other news, I've got my Java save loader auto-detecting the source game, so there's that.
2Tie

Rumours of my death have been greatly exaggerated.
Level: 57


Posts: 405/920
EXP: 1476632
For next: 9296

Since: 02-22-10
From: The Codrex, Karda Nui

Since last post: 5 days
Last activity: 59 min.

Posted on 02-26-14 09:26:37 PM Link | Quote
*clapsclaps* Good job, I woulda never caught that heh. Yay progress!
GuyPerfect
Catgirl
Level: 64


Posts: 925/1093
EXP: 2211411
For next: 2686

Since: 07-23-07


Since last post: 101 days
Last activity: 59 days

Posted on 03-11-14 02:02:33 PM Link | Quote
So prior to yesterday, my last day off of work was February 26. |-:

But I've been busy on my free time and successfully documented the Pokédex data for generation 3. Of course it wasn't a simple matter of bit flags, but I got it sorted out and I can get back to work on the migration utility now.

On a side, thank the stars for VBA's fast-forward button. I had to play through all of Emerald and LeafGreen to unlock the National Pokédex and compare what changed from the previous save. It was easier with Sapphire, since all you have to do to unlock National mode is make a trade.
DoktorGilda
Random nobody
Level: 4


Posts: 1/3
EXP: 189
For next: 90

Since: 03-09-14


Since last post: 3.0 years
Last activity: 3.0 years

Posted on 03-12-14 12:00:26 AM Link | Quote
That's incredible, I'm really happy to see your findings and progress on this! I'm looking forward to seeing your next updates and using the program if/when you distribute it.
Pages: 1 2 3 4 5Next newer thread | Next older thread
Jul - Game Research/Hacking/Modding - Migrating Pokémon from Generation 2 to 3 New poll - New thread - New reply




Rusted Logic

Acmlmboard - commit 2f1bc75 [2017-08-27]
©2000-2017 Acmlm, Xkeeper, Inuyasha, et al.

28 database queries, 15 query cache hits.
Query execution time: 0.151183 seconds
Script execution time: 0.030515 seconds
Total render time: 0.181699 seconds