Monday, December 29, 2008

Even more on magic resist rates

(Edit - Dec. 30: Some further thoughts on the enspell experiment.)

In the past week, I discussed over several posts a Japanese player's extensive exploration of magic resist rates, specifically changes in "magic hit rate" ("lack of resist" rate) with each of several controllable factors (use of elemental staves, elemental magic skill, INT, and magic accuracy) for nukes alone. You may review these posts under the "magic resist analysis" tag.

I would've left it at that, but I didn't realize until now that there was a "reaction" on BG forums generated by my discussion of lodeguy's data. In particular, there are a few data sets that I would like to go over as they may help focus further investigation.

You may skip to the summary if you like.

Alkalurops vs. HQ elemental staff

As a competitor to elemental staves, Alkalurops seems to be maligned from an accuracy standpoint because it's assumed that its "effective" accuracy (comprised of contributions from INT/MND/CHR +10 and magic accuracy +20) is worse than that of a HQ elemental staff, usually because it is assumed that staves provide a multiplicative accuracy bonus (in the absence of any real evidence). (Obviously, for nukes Alkalurops is inferior to HQ staves merely from a damage standpoint.) But, if you've read any of my recent posts, you should now be comfortable asserting that INT does make an important contribution to reducing resist rates, at least when it comes to nukes.

Consider the results of the following experiment comparing the accuracy of Alkalurops to that of Terra's Staff (check the forum post for details as I am not interested in rehashing experimental conditions):


ConditionNo resistSome resist
No staff
296
(.296)
704
(.704)
Terra's Staff
354
(.443)
446
(.558)
Alkalurops
355
(.444)
445
(.556)


At the risk of committing a Type II error, this result is not all that surprising given what we've inferred thus far about magic hit rate bonuses from elemental staves, magic accuracy, and INT.

The Terra's Staff (HQ) seems to be providing a constant 15% increase to the "success" (no resist) rate, which could be considered an elemental magic accuracy bonus of +30 cut in half (+15) since the initial and final magic hit rates are both under 50%.

A possible explanation for the Alkalurops is that it seems to be providing an "effective" magic accuracy bonus of +30, with contributions from its magic accuracy attribute (magic accuracy +20) and from its INT attribute (INT +10 corresponding to an constant increase of 10% hit rate since ΔINT is below +10). This effective accuracy bonus is cut in half since the initial and final magic hit rates are both under 50%.

Not only does this example suggest that the accuracy effect of added INT (like magic skill, magic accuracy, and staff accuracy bonuses) is attenuated below 50% magic hit rate, it also suggests that Alkalurops is a strong enfeebling staff and an acceptable replacement for a whole family of HQ elemental staves (again, when it comes to enfeebling). When it comes to enfeebles, it would not be that great a leap to conclude that, based on results from nuking, INT/MND/CHR must provide some accuracy bonus in addition to a potency bonus (where applicable). The next example shows that additional MND does reduce the "complete resist" rate of Paralyze.

Experiments with Paralyze

This next data set comes from "FFXI Hunter's Bible Version II" and is the result of 8,000 casts of Paralyze on level 84 Aura Statues (were they sure the level was fixed?) under varying conditions. The event of "success" was defined as anything that wasn't a complete resist (both non-resists and partial resists). A summary of point and interval estimates for this data set follows:

                  Analysis Of Parameter Estimates

Standard Wald 95% Confidence
Parameter Estimate Error Limits Pr > ChiSq

Intercept -1.9919 0.4107 -2.7969 -1.1870 <.0001
skill 0.0064 0.0013 0.0038 0.0090 <.0001
macc 0.0073 0.0013 0.0046 0.0099 <.0001
mnd 0.0075 0.0013 0.0048 0.0101 <.0001
staff HQ 0.2070 0.0207 0.1665 0.2475 <.0001
staff NQ 0.1710 0.0205 0.1308 0.2112 <.0001
day Ice 0.0160 0.0192 -0.0216 0.0536 0.4048
day Fire -0.0210 0.0187 -0.0576 0.0156 0.2610


I know it is kind of trivial to give such a summary (the result of fitting the saturated linear probability model) when you can inspect the data directly and see that MND does affect the accuracy of Paralyze, but it does conveniently summarize the precision of these point estimates in red.

It is important to note that the given point estimates are not appropriate to describe the actual changes in magic hit rate (no-resist rate) because they also encompass partial resists. These estimates should therefore be higher than the "real" no-resist rates.

The other important observation is that the effects of Iceday and Firesday, respectively, on the accuracy of Paralyze are not even close to being statistically significant (at the 5%, 10%, 15%, and 20% levels), if they even exist at all. The relevant p-values are in red. It is not mentioned whether an obi was used.

The following is a convoluted discussion on whether there are discrete levels of Paralyze resists and how they may be observed indirectly. You can skip this part since it's mainly for my own amusement.

We know (or should know from experience) that levels of resists for Sleep, Poison, and the "elemental enfeebling" line of spells seem to have 3 distinct levels of resists (full duration, half-duration, and complete resist). Is it appropriate to conclude that enfeebling spells with a "continuous" range of durations, like Paralyze (only Paralyze?), also have discrete levels of resistance?

If the point estimates above are really the result of Paralyze resists following a multinomial distribution, it may help to illustrate what these point estimates might really be... estimating. For example, if many enfeebling spells follow a multinomial distribution, using the exact same logic of conditional probability that has been validated for nukes (not to say that this is easily verifiable for enfeebles, because it would take forever to do so; therefore, I cannot say whether this assumption is valid at the moment), then the multinomial proportions may vary with some level of "input" (INT, magic skill, magic accuracy, whatever) as follows:



Here, I describe a situation where there are only 3 levels of resists (no resist, half-resist, and full resist). This image roughly describes how p, the probability of no resist, and p(1 - p), the probability of a half-resist, may vary with input level when p is below 50%.

Suppose that the quantity p + p(1 - p) (illustrated in red) is what the point estimates above are actually trying to describe. Specifically, the point estimates would then be estimates of the slope (rate of change) of the curve in red. They seem "plausible" enough considering the precision (or imprecision) of these estimates compared to the theoretical rate of change of this curve as illustrated below:



So, it may really be the case that there are discrete levels of Paralyze resists. It's just that they seem difficult to observe directly, and may be observed indirectly by obtaining sample proportions of no resists and partial resists summed together (the sum being what is easily observed).

Obviously, the theoretical slope of p + p(1 - p) is not constant. If this applies to enfeebles such as Sleep, et al., then any experiments using Sleep and the like must account for this. (Not that anyone does or would.)

What if there are four levels of resists for Paralyze (and other enfeebling magic spells)? I also generated graphs for this case:



And for the rates of change:



Obviously, these apply to nukes. You can use lodeguy's data to verify that these trends apply to elemental magic skill and INT.

Using enspells to estimate changes in "magic hit rate" with magic accuracy

This approach seems to be a clever way to accumulate a large number of "trials" somewhat easily because it takes advantage of auto-attack and there are discrete levels of resists that are easily observed (if not automatically tallied with a parser). So, you can possibly deduce (estimate) a target's magic evasion after controlling for your own magic accuracy "score" and perhaps level correction/penalty.

Note that since the author has ice accuracy merits, the recorded magic accuracy levels reflect that. (I missed that initially.) But throwing caution to the wind, simple analysis of the above data (modeling the "full" enspell rate, or no-resist rate) cranks out the following results:

           Criteria For Assessing Goodness Of Fit

Criterion DF Value Value/DF

Deviance 2 0.0042 0.0021
Scaled Deviance 2 0.0042 0.0021
Pearson Chi-Square 2 0.0042 0.0021
Scaled Pearson X2 2 0.0042 0.0021
Log Likelihood -1678.4884


Algorithm converged.


Analysis Of Parameter Estimates

Standard Wald 95% Confidence Chi-
Parameter DF Estimate Error Limits Square Pr > ChiSq

Intercept 1 -0.8225 0.2504 -1.3133 -0.3316 10.79 0.0010
skill 1 0.0048 0.0009 0.0030 0.0066 28.05 <.0001
macc 1 0.0031 0.0051 -0.0069 0.0132 0.37 0.5428
element earth 1 0.0950 0.0239 0.0481 0.1419 15.77 <.0001
element ice 1 0.1418 0.0721 0.0004 0.2831 3.86 0.0493


If I entered the data correctly (rather, if the author bothered to record his data correctly), the goodness-of-fit statistics make the data appear very suspicious (p-value of .9979022 for the deviance statistic). If the estimates can be trusted, it appears that a one-point increase in enhancing magic skill increases the probability of a "full" enspell by almost .005.

The difference in magic accuracy levels is small (only 5) so it will be hard to quantify the effect of magic accuracy precisely without increasing sample sizes.

Perhaps the accuracy of enspells is handled in a fundamentally different way than other types of magic are. (We see that the damage of enspells may be increased by adding enhancing magic skill, whereas the damage of nukes is not directly affected by increases in elemental magic skill.) It would then be a waste of time to reconcile these results to lodeguy's. On the other hand, it is much easier to investigate enspells for the reasons cited earlier.

Summary

Some "salient" observations:

  1. In light of previous observations concerning the effects of INT and magic accuracy on magic hit rate ("lack of resist" rate), some evidence suggests that Alkalurops may be comparable to HQ elemental staves in terms of effective magic accuracy.
  2. MND affects the accuracy of Paralyze.
  3. Neither Firesday nor Iceday seems to affect the accuracy of Paralyze.
  4. Know what you are measuring when investigating magic resist rates with enfeebles. Just because an enfeebling spell doesn't resist completely doesn't mean it wasn't a partial resist.
  5. Investigation of enspells can lead to further insights about magic resist rates.

No comments:

Post a Comment