More on magic resist rates

(Edit - Dec. 30: First image was fixed.)

(Edit - 7:00 PM PST: I wrote the last section in a muddle and it makes no sense. It was amended.)

(Edit - 5:00 AM PST: Summary added.)

This post is a continuation of my discussion of extensive data that a Japanese blogger collected for the purposes of investigating the relationship between "magic hit rate"--defined as a "lack of resist" rate for the purposes of my discussion unless otherwise stated--and each of several factors that are known to affect the accuracy of magic spells.

So far, I have gone over the possible relationship between magic hit rate and elemental staves and the relationship between hit rate and elemental magic skill. You may view the "tentative" conclusions so far. (I say "tentative" because I will be the first to acknowledge the limited scope of the binary regression models that are the basis for making any conclusions.) I will continue to focus exclusively on the accuracy of direct-damage magic ("nukes") as opposed to other magic types (but I may get around to discussing enfeebling magic later).

The importance of checking for linearity

First off, I just want to make a few comments regarding the (apparently) piecewise-linear relationship (which is plausible because it fits the data well, even if the author's procedure was more of an ad hoc one... not sure) between magic hit rate and elemental magic skill that the blogger described.

Yes, in the past I have said it may be feasible to estimate changes in "magic hit rate" with elemental magic skill by choosing two levels of elemental magic skill that are very far apart (and hope that your magic hit rate isn't capped before your higher level), and then perform some "regression" procedure, which is basically drawing a line through the two observed rates (sample proportions). If you fix the number of observations you will set out to collect, allocating your number of observations equally between the two levels will be the most efficient way to detect an effect (a statistical power rationale). Obviously, though, you can't even check for the linearity assumption (hence the term linear regression) since a line through two observed values is a perfect fit, and if the trend is not linear overall, the validity of your point estimate is highly suspect.

As an example, I return to this data set (experimental conditions: Water magic on a lv78 Earth Elemental, using 78 INT and varying levels of elemental magic skill with a Neptune's Staff):

Skill	No resist	1/2 resist	1/4 resist	1/8 resist
230	1233 (.380)	768 (.237)	499 (.154)	746 (.230)
240	832 (.434)	469 (.245)	245 (.128)	369 (.193)
250	1536 (.476)	826 (.256)	399 (.124)	468 (.145)
262	1000 (.598)	373 (.223)	163 (.097)	137 (.082)
270	1780 (.667)	600 (.225)	188 (.070)	99 (.037)

Performing a regression on this data yields the following results:

           Criteria For Assessing Goodness Of Fit

Criterion                 DF           Value        Value/DF

Deviance                   3         19.8838          6.6279
Pearson Chi-Square         3         19.8790          6.6263

         Analysis Of Parameter Estimates

                     Standard  Wald 95% Confidence
Parameter  Estimate     Error        Limits

Intercept   -1.2832    0.0719  -1.4242     -1.1422
Skill        0.0072    0.0003   0.0066      0.0077

As I said previously, obviously this model, which assumes a linear relationship between hit rate and skill over the entire range of skill, is a poor fit to the data. The Japanese blogger was aware of this and proposed piecewise linearity. I suspect a failure to check for lack of fit is behind the estimated hit rate increases described on wiki.ffo.jp for 1 point of elemental magic skill (.064), 1 point of INT/MND/CHR attribute (.074), and 1 point of magic accuracy (.072), although there is no source cited.

For your convenience, I have furnished a graph plotting the observed magic hit rates (sample proportions) versus elemental magic skill for the above data set, and plotted the linear probability model fit to show poorness of fit. I also drew 95% (exact) confidence intervals for the point estimates:



I did include a loglinear model fit mainly for my own amusement (not as bad a fit), but there is no reason to think "the dev team" would really use some kind of explicit loglinear relationship (much less some general logistic one) for anything in FFXI. So I lean toward piecewise linearity because that would be simple to implement, I think.

I must admit, however, that I'd like to see any trends beyond 270 elemental magic skill and below 230 elemental skill, but this thought only comes about because the blogger was perceptive enough to propose piecewise linearity. Is there even enough elemental skill equipment out there to test across a range of elemental skill broader than a 40-point range?

That said, we can proceed to examine the relationship between magic hit rate and INT, keeping in mind the perils of assuming "global" linearity.

Magic hit rate versus INT

Refer to the original post for specifics. (There is more commentary, but the table doesn't make any sense to me.) The author endeavored to examine the relationship between magic hit rate versus INT for a wide range of ΔINT (his INT minus the target's INT). Did he suspect "global" nonlinearity with ΔINT to begin with, or did these results lead him to suspect a similar trend with elemental magic skill? You'll have to ask him.

Level 78 Earth Elementals appear to have 73 INT. He used some Water nuke with a Neptune's Staff with 262 elemental magic skill. The data is summarized as follows:

ΔINT	No resist	1/2 resist	1/4 resist	1/8 resist
-20	957 (.545)	439 (.250)	183 (.104)	176 (.100)
-15	1000 (.598)	373 (.223)	163 (.097)	137 (.082)
-10	637 (.653)	230 (.236)	73 (.075)	36 (.037)
-5	870 (.678)	270 (.210)	101 (.078)	42 (.033)
0	886 (.721)	242 (.197)	71 (.058)	30 (.024)
+10	1585 (.821)	287 (.149)	43 (.022)	15 (.008)
+20	884 (.854)	129 (.125)	19 (.018)	3 (.003)
+30	1387 (.909)	127 (.083)	9 (.006)	3 (.002)

I don't really feel like replicating graphs that the original author already created, so I will just show you the one he created plotting the data (with 68% confidence intervals) and his piecewise linear regression:



Perhaps the presence of the piecewise regression model fit influences your perception of the the trend. Still, it seems that INT appears somewhat less effective at high levels of ΔINT.

Thus, it is pretty obvious that assuming "global" linearity would yield a poor fit to the data, so the piecewise linear regression (whether the cutoff point is intuited or rigorously chosen) approach seems reasonable in order to estimate precisely the effect of a 1-point change in INT on magic hit rate (depending on the range of ΔINT).

And whether or not you use ordinary least-squares regression (which assumes a normal response, which a marginally binomial proportion is not) or a MLE method (GLM), the point and interval estimates of the slopes are pretty close anyway. The following uses MLE estimation for ΔINT between -20 and 10:

           Criteria For Assessing Goodness Of Fit

Criterion                 DF           Value        Value/DF

Deviance                   4          1.6641          0.4160
Pearson Chi-Square         4          1.6626          0.4157

         Analysis Of Parameter Estimates

                     Standard  Wald 95% Confidence
Parameter  Estimate     Error        Limits

Intercept    0.7291    0.0051   0.7190      0.7391
dint         0.0090    0.0004   0.0082      0.0098

It seems that between ΔINT -20 and ΔINT 10, a 1-point increase in INT is expected to result in a 0.9% increase in magic hit rate. (Aside: I dislike expressing changes in proportions--hit rate is a proportion--as percentages because they are often interpreted as increases by a factor of (1+[percent]/100), which is not what I mean. So that is why I usually lean toward expression of rates in decimal form... not that it really helps understanding all that much.)

Is .009 (0.9%) significantly different (statistically) than .01 (1%)? The 95% confidence interval bounding the true rate of change in magic hit rate happens to exclude .01 (1%), so yes. But of course, it's possible Type I error has manifested.

Finally, considering the range of ΔINT between 10 and 30:

           Criteria For Assessing Goodness Of Fit

Criterion                 DF           Value        Value/DF

Deviance                   1          0.8059          0.8059
Pearson Chi-Square         1          0.8156          0.8156

         Analysis Of Parameter Estimates

                     Standard  Wald 95% Confidence
Parameter  Estimate     Error        Limits

Intercept    0.7741    0.0133   0.7482      0.8001
dint         0.0044    0.0006   0.0033      0.0056

It seems that between ΔINT 10 and ΔINT 30, a 1-point increase in INT is expected to result in a .0044 increase in magic hit rate. But we cannot distinguish between .0045 and .005 given the 95% confidence interval.

Observations: I am interested in what happens to the effect of INT below 50% magic hit rate. This could be achieved by removing the Neptune's Staff and repeating the experiment. (Have fun collecting 11,000 observations!) But this data set (not mine) does not show any effect of INT+30 (is ΔINT after INT+30 still below 0 for an Elvaan mage versus an Ebony Pudding?) at 242 elemental magic skill. Is this because the "base" magic hit rate (whatever it is) is well below 50%?

Conclusion: Above 50% "base" magic hit rate (whatever it is), it appears that below ΔINT 10, 1 point of INT gives about a .01 increase (or .009 if you are a stickler for statistical significance) in magic hit rate for direct-magic damage, and above ΔINT 10, a .005 increase in magic hit rate.

I hope this result can be generalized to any kind of mob and also to MND and CHR.

Magic hit rate versus magic accuracy

I don't see an in-depth examination of the effect of magic accuracy (from equipment). Early on, he seemed to have been trying to get a feel for things (see the original post). I just see the following data pertaining specifically to magic accuracy:

For level 75 Qiqirn Archaeologists (Aydeewa Subterrane), using Stone magic, 82 INT, and 230 elemental magic skill, and no elemental staff (1,365 observations):

Condition	No resist	1/2 resist	1/4 resist	1/8 resist
baseline	379 (.420)	194 (.215)	133 (.147)	196 (.217)
+10 m. acc	205 (.443)	124 (.268)	58 (.125)	76 (.164)

For level 75 Steelshells (The Boyahda Tree), using Stone magic, 82 INT, and 230 elemental magic skill, and no elemental staff (1,142 observations):

Condition	No resist	1/2 resist	1/4 resist	1/8 resist
baseline	580 (.744)	141 (.181)	49 (.063)	10 (.013)
+10 m. acc	303 (.837)	49 (.135)	8 (.022)	2 (.006)

Since Qiqirn are resistant to earth magic, there is a huge discrepancy in the magic hit rate of Stone between the two sets of trials.

For the Qiqirn trial, the magic accuracy effect is not statistically significant, but that may just be a consequence of "small" sample sizes (poor statistical power to detect an effect size so small):

                   Analysis Of Parameter Estimates

                     Standard  Wald 95% Confidence    Chi-
Parameter  Estimate     Error        Limits         Square  Pr > ChiSq

Intercept    0.4202    0.0164   0.3880      0.4524  653.65      <.0001
macc         0.0023    0.0028  -0.0033      0.0078    0.64      0.4254

For the Steelshell trial, the magic accuracy effect is highly, statistically significant, but the interval estimate is rather wide:

                    Analysis Of Parameter Estimates

                     Standard  Wald 95% Confidence     Chi-
Parameter  Estimate     Error        Limits          Square  Pr > ChiSq

Intercept    0.7436    0.0156   0.7129      0.7742  2262.00      <.0001
macc         0.0093    0.0025   0.0045      0.0142    14.05      0.0002

Still, in light of what we know about the relationship between magic hit rate and each of the factors that have been investigated well above 50% magic hit rate and well below 50% magic hit rate (elemental staff and elemental magic skill), it seems reasonable to infer that 1 point of magic accuracy is equivalent to about 0.5% magic hit rate below 50% "base" magic hit rate and about 1.0% magic hit rate, at best, above 50% "base" magic hit rate. (The confidence interval is duly noted, but common sense dictates that the 1-point magic accuracy bonus is 1% hit rate at best.)

What the heck is "base" magic hit rate, and what evidence supports such an idea?

I am speculating that "base" magic hit rate is the result of a calculation that compares your "magic accuracy" score before equipment and buffs (debuffs) to a mob's "magic evasion" score, which may be comprised of elemental resistance factors.

So far, the main purpose of making a distinction between a "base" magic hit rate and magic hit rate bonuses from equipment (and possibly buffs/debuffs) is that the bonuses from staves, elemental magic skill, and magic accuracy (and probably INT) are conditional on magic hit rate, based on the extensive data provided. And how do you go about determining the bonuses from equipment if the bonuses from equipment determine the "base" hit rate?

My initial thought was that if a "base" hit rate is below 50%, then any bonuses from equipment will be as I described previously, even if the actual hit rate ends up being above 50%. Again, speculation.

As far as evidence goes, here is one that contradicts what I just wrote. Yet another post from our highly esteemed Japanese blogger illustrates the magic hit rate bonus from using a staff that is the same element as that of the magic being used. The data are summarized as follows:

For level 75 Qiqirn Archaeologists (Aydeewa Subterrane), using Stone magic, 82 INT, and 230 elemental magic skill, and no elemental staff (1,307 observations):

Staff	No resist	1/2 resist	1/4 resist	1/8 resist
None	379 (.420)	194 (.215)	133 (.147)	196 (.217)
Terra's Staff	262 (.647)	81 (.200)	29 (.072)	33 (.081)

Note that the interval estimate of the magic hit rate without staff (not shown) does not cover .50, so I am 95% confident the real hit rate is below .50. Furthermore, the interval estimate of the magic hit rate with Terra's Staff (also not shown) does not cover .50 either, so I am 95% confident that the magic hit rate with Terra's Staff is well above .50.

Previously it was shown that a 95% confidence interval for the HQ staff effect "well" below 50% hit rate was (.1359, .1665). Here, the point estimate for the staff bonus appears to be .227, but how precise is this estimate?

            Analysis Of Parameter Estimates

                           Standard  Wald 95% Confidence
Parameter        Estimate     Error        Limits

Intercept          0.4202    0.0164   0.3880      0.4524
staff      HQ      0.2267    0.0289   0.1701      0.2833
staff      None    0.0000    0.0000   0.0000      0.0000

What's going on? First, one set of data showed that for magic hit rates above 50%, a HQ staff seemed to confer (what is thought to be) a constant 30% magic hit rate bonus (estimated). Then, another set of data showed that for magic hit rates below 50%, a HQ staff seemed to confer (what is thought to be) a constant 15% magic hit rate bonus (estimated). But the above 95% CI covers neither .15 nor .30.

So this data seems to undermine the idea of the "base" hit rate check I speculated about, unless a transition below 50% magic hit rate to above 50% magic hit rate (and vice versa) is handled by the game in a way that is difficult to observe. (Well, I have gone delirious at this point, so let me revisit this later.)

Summary

The conclusions inferred from the data so far (see my last post as well for a summary) rest on a few ideas and concessions that really warrant further examination:

• There are two distinct "regimes" of magic hit rate before any bonuses from equipment (and probably buffs/debuffs and food, etc.) that determine the magnitude of the accuracy bonuses from elemental magic skill, elemental staves, and magic accuracy (all from equipment).
• One region is below 50% magic hit rate
• The other region is above 50% magic hit rate
• We are assuming piecewise linearity to model the existence of the above phenomenon. Otherwise, some nonlinear relation (e.g. logistic) will result in more complex interpretations
• I acknowledge that only direct-magic damage ("nukes") was investigated. "Further examination" here means that we should look at other types of magic (enfeebling) to see if the conclusions for direct-magic damage can be generalized.
• I concede the possibility of weather/day possibly confounding the results. But these effects do not process 100% for the magic damage calculation, so if they also apply to magic accuracy, the effect is probably not 100% either (without obis). The effect may also be weak and hard to detect, if it even exists at all; if this is the case, it is not a serious confounding threat. (I don't see any data to corroborate this though. You can perform some regression diagnostics to check for omitted explanatory variables, too.)

However, even if the model described above is not exactly as SE designed magic accuracy/magic hit rate to work, it still is a model that seems to approximate well the "reality" of the situation (for nukes). It's not like I have a vested interest in promoting this view of magic hit rate bonuses. It is well within the realm of possibility that the data provide only a limited view of the whole situation.

That said, so far it appears (and I do emphasize that these are estimates) that, given the data so far:

• If the initial and final magic hit rates are both below 50%, then
• An HQ staff of the correct element gives a constant increase of 15% magic hit rate
• A NQ staff of the correct element gives a constant increase of 10% magic hit rate
• 1 point of elemental magic skill gives a constant increase of 0.5% magic hit rate
• 1 point of magic accuracy gives a constant increase of 0.5% magic hit rate (caveat being the evidence is not that strong)
• If the initial and final magic hit rates are both above 50%, then
• An HQ staff of the correct element gives a constant increase of 30% magic hit rate
• A NQ staff of the correct element gives a constant increase of 20% magic hit rate
• 1 point of elemental magic skill gives a constant increase of 1% magic hit rate
• 1 point of magic accuracy gives a constant increase of 1% magic hit rate (caveat being the evidence is not that strong)


"Open question": If 50% magic hit rate (before equipment or "base") really is a critical point, what happens to accuracy bonuses (or penalties) that cross this critical point?

Finally, we also saw that for INT, specifically ΔINT, the difference between your INT and your target's INT:


• If the initial and final magic hit rates are both above 50%, then
• Between ΔINT -20 and ΔINT 10, 1 point of INT gives a constant increase of 1% magic hit rate
• Between ΔINT 10 and ΔINT 30, 1 point of INT gives a constant increase of 0.5% magic hit rate
• There is no information for initial and final magic hit rates both below 50% magic hit rate
  


"Open question": Suppose ΔINT 10 really is a critical point. Then what happens to INT bonuses (or penalties) that cross this critical point? (The Burn experiment that the Japanese blogger described, which I did not address, seems inconclusive on this point.)

Finally, there appears to be a level correction/penalty to magic hit rate when targeting something higher level than you.

The temptation now (at least for me) is to seek out existing data sets and see if they are consistent with the model just described, but I will try to look at the "open questions" I just identified in a later post.

On magic resist rates

(Edit (Dec. 30): Obvious mistake when talking about tests for two proportions.)

(Edit (Dec. 18): I added sample proportions in the tables I presented to add a little clarity to the presentation. The sample proportions in each row of a table are those for a multinomial distribution, conditional on the level of the factor described in the left-most cell, so the proportions sum to 1 row-wise.)

I recently came across a Japanese player's blog with many interesting insights on "magic hit" rate ("lack of resist" rate) supported by data that apparently he himself collected. I didn't collect this data. I do not take credit for any of this. I have no idea how these observations are regarded in the Japanese FFXI "community," but they don't seem to be promulgated on wiki.ffo.jp, one of a few Japanese analogues to FFXIclopedia.

At any rate, assuming the data are legitimate, this blogger deserves a lot of credit for investigating the relationship between magic hit rate and each of several controllable factors (staff, elemental magic skill, INT). The following discussion summarizes the blogger's findings, leavened by my own observations. (Since I don't know Japanese, I may be glossing over any qualifying statements the blogger may be making about his data, but his presentation of his main conclusions is straightforward, a stark contrast to my rambling, invective-laden posts.)

Note: for the purposes of this post, "magic hit rate" is synonymous with "no resist rate" (in other words, no partial resists). If you want to skip all the rambling that follows, you can jump to the summary.

The magic hit rate bonus of elemental staves

After casting a Water spell (presumably the same one) on a level 78 Earth Elemental 7,990 times (!) with 103 INT, 270 elemental magic skill, and varying levels of staff effect (no staff, NQ staff, HQ staff), the blogger obtained the following results:

	No resist	1/2 resist	1/4 resist	1/8 resist
No staff	2324 (.598)	906 (.233)	385 (.099)	271 (.070)
Water Staff	1594 (.793)	336 (.167)	51 (.025)	29 (.014)
Neptune's	1858 (.887)	204 (.097)	30 (.014)	2 (.001)

Modeling the "no resist" rate versus staff (levels: no staff, NQ, and HQ), the following estimates are obtained:

            Analysis Of Parameter Estimates

                           Standard  Wald 95% Confidence
Parameter        Estimate     Error        Limits

Intercept          0.5980    0.0079   0.5826      0.6135
staff      HQ      0.2893    0.0105   0.2687      0.3098
staff      NQ      0.1950    0.0120   0.1715      0.2185
staff      None         0    0.0000   0.0000      0.0000

We are 95% confident that a HQ staff (co-aligned in element with that of a direct-damage magic spell) provides an absolute increase of magic hit rate (or resist rate decrease) between .2687 and .3098. We are also 95% confident that the NQ staff provides an absolute increase of magic hit rate between .1715 and .2185.

Observations: Is the magic hit rate bonus of elemental staves additive (constant increase) or multiplicative (percent increase)? I am not going to pretend to know the exact details since we cannot know them. However, given the blogger's efforts at obtaining precise point estimates, we can just model the bonus as a static increase, and that's good enough for me. It's not like anyone can actually verify or reject this modeling assumption in normal gameplay.

This blogger seems to be making a point about the experiment being conducted above 50% hit rate without a staff bonus. In previous posts (which I will get to later), he shows evidence of a discontinuity in the rate of change in magic hit rate (sounds funny, right?) versus elemental skill around 50% hit rate. Actually, this observation could explain some curious trends in this data set that I mentioned before, such as that INT seems to have no effect at 242 elemental skill, but the success rate is well below 50%. Perhaps any INT increase makes zero contribution conditional on the overall hit rate being below 50%? It seems pretty weird to condition on an outcome if you ask me...

It may be interesting to note that the probability distribution of resists for elemental magic, given some level of magic accuracy (w/ staff or not), could be modeled as follows (yes... I will attribute this to the same blogger... source):

no resist: p
1/2 resist: p(1-p)
1/4 resist: p(1-p)²
1/8 resist: (1-p)³

Here, p is the probability of "success" (no resist).

I mention this only because one of the more trivial complaints I've heard about estimating the staff bonus is that the full range of resists is not accounted for. Well, if the distribution of resists is determined by only one parameter, and the estimated change in that parameter (rate of no resists) is used to estimate the staff bonus, there is no problem. Perhaps this isn't the case with enfeebling magic (well, it isn't with the Sleep family of spells, as there is a full resist, half resist, and no resist, as far as I can tell), but it looks like a pretty good fit to the above data. Also, I don't think SE would bother assigning a unique parameter to each resist level (under the restriction that they all add to 1).

Conclusion: Above .50 magic hit rate, I would provisionally treat the HQ staff bonus as a .30 magic hit rate increase. The NQ staff bonus appears to be a .20 magic hit rate increase.

Apparently, wiki.ffo.jp does report a bonus of +20 magic accuracy for NQ staves and +25-30 for the HQ. I don't know if those values were obtained independently or not. I haven't really emphasized it until now, but I don't think magic accuracy is synonymous with hit rate until shown otherwise. In other words, there is no reason for me to assume that magic accuracy has the same effect regardless of "base" magic hit rate level.

Magic hit rate versus elemental magic skill

After casting a Water spell (presumably the same one) on a level 78 Earth Elemental 12,730 times (!!) with 78 INT and varying levels of elemental magic skill (with a HQ staff), the blogger obtained the following results:

Skill	No resist	1/2 resist	1/4 resist	1/8 resist
230	1233 (.380)	768 (.237)	499 (.154)	746 (.230)
240	832 (.434)	469 (.245)	245 (.128)	369 (.193)
250	1536 (.476)	826 (.256)	399 (.124)	468 (.145)
262	1000 (.598)	373 (.223)	163 (.097)	137 (.082)
270	1780 (.667)	600 (.225)	188 (.070)	99 (.037)

This data set illustrates the importance of checking for lack of fit when regressing a response (magic hit rate) on some predictor variables (elemental magic skill). If you apply the linear probability model for the above data set, you will get the following deviance and Pearson chi-square goodness-of-fit statistics, indicating an extremely poor fit to the data:

           Criteria For Assessing Goodness Of Fit

Criterion                 DF           Value        Value/DF

Deviance                   3         19.8838          6.6279
Pearson Chi-Square         3         19.8790          6.6263

The author observes that the relationship between magic hit rate and elemental magic skill seems to be piecewise linear (see post for graph), which accounts for the lack of fit across the whole range of elemental magic skill. I am inclined to agree.

The author then collected data for the same experimental conditions but without a HQ staff:

Skill	No resist	1/2 resist	1/4 resist	1/8 resist
230	521 (.234)	387 (.173)	283 (.127)	1040 (.466)
240	459 (.267)	361 (.210)	235 (.137)	666 (.387)
250	732 (.331)	470 (.213)	316 (.143)	691 (.313)
260	608 (.382)	412 (.259)	228 (.143)	345 (.217)
270	916 (.426)	529 (.246)	314 (.146)	390 (.181)

It is interesting to note that if you model magic hit rate, combining the two data sets just presented (one with HQ staff, one without), one obtains the following estimates applying the linear probability model (the scope of the model is between 230 and 250 elemental magic skill):

           Analysis Of Parameter Estimates

                          Standard  Wald 95% Confidence
Parameter       Estimate     Error        Limits 

Intercept        -0.8782    0.1078  -1.0896     -0.6668
skill             0.0048    0.0005   0.0039      0.0057
staff      Yes    0.1512    0.0078   0.1359      0.1665
staff      No     0.0000    0.0000   0.0000      0.0000

(Note: It is possible to test for interaction between the skill and staff effects, or, in other words, whether the staff effect is multiplicative or additive. If multiplicative, the staff bonus should vary with elemental skill. I did so, and the interaction effect is not significant. But, if the staff bonus does vary with skill, it will be very hard to detect statistically if a hypothesized percent increase is not so different from a constant increase.)

We are 95% confident that, controlling for the staff effect, a one-point increase in elemental magic skill provides in an absolute increase of magic hit rate between .0039 and .0057 below 50% magic hit rate.

We are also 95% confident that, controlling for elemental magic skill, a HQ staff (co-aligned in element with that of a direct-damage magic spell) provides an absolute increase of magic hit rate (or resist rate decrease) between .1359 and .1665 below 50% magic hit rate.

Unfortunately, I cannot perform the same analysis for hit rate observations above 50%, both without and with a HQ staff, since the author did not provide the necessary data. However, I can still regress hit rate on elemental magic skill. (The experimental setup is the same except that 103 INT was used along with a HQ staff.)

         Analysis Of Parameter Estimates

                     Standard  Wald 95% Confidence
Parameter  Estimate     Error        Limits

Intercept   -1.7053    0.1604  -2.0197     -1.3909
Skill        0.0096    0.0006   0.0084      0.0108

(Note: the scope of the model is between 250 and 270 skill.)

We are 95% confident that a one-point increase in elemental magic skill provides in an absolute increase of magic hit rate between .0084 and .0108 above 50% magic hit rate.

Observations: Interesting! It seems the elemental staff bonus is conditional on what the magic hit rate is, presumably before the staff bonus is applied. And, at least under 50% hit rate, 1 point of elemental magic skill appears roughly equivalent to .005 magic hit rate.

Yet above 50% hit rate, 1 point of elemental magic skill appears roughly equivalent to .01 magic hit rate. I wonder if magic accuracy also exhibits a similar trend before 50% hit rate and after 50% hit rate. Then perhaps the increase in hit rate from staves is equivalent to a magic accuracy increase. But how does a "check" on a calculated hit rate work if the effects of skill and accuracy (and staff) depend on the hit rate? Seems pretty circular to me.

What if you are near this critical point and then add, say, +15 elemental magic skill from a piece of equipment? Perhaps the "base" hit rate calculation depends on "native" magic skill compared to some target's "magic evasion." So it may be desirable to maximize one's elemental magic skill, for example, so that your base hit rate relative to some target of interest is above 50%. Otherwise, your equipment bonuses will be checked against a hit rate below 50%, and you will be penalized accordingly. This is all speculation though.

Conclusion: The data suggest the following:

Below .50 "base" magic hit rate:
HQ staff bonus: +.15 magic hit rate (perhaps +15 magic accuracy)
NQ staff bonus: +.10 magic hit rate (inferred)
elemental magic skill: +1 skill is equivalent to .005 hit rate

Above .50 "base" magic hit rate:
HQ staff bonus: +.30 magic hit rate (perhaps +30 magic accuracy)
NQ staff bonus: +.20 magic hit rate (inferred)
elemental magic skill: +1 skill is equivalent to .01 hit rate

Magic hit rate and possible level correction

One of this blogger's earlier posts appears to address the possibility of a level penalty when a target is higher level than you.

The targets of interest were level 75 Steelshells (even match) and level 76 Steelshells (tough). Presumably, both types of steelshells have the same INT and "magic evasion," although for practical purposes one cannot really distinguish between "pure" magic evasion and a reduction in magic hit rate from a level correction. At any rate, there is a data set to test for a possible effect of level difference.

Note that the author also recorded observations versus day of the week. Considering that any day (or weather) effect processes relatively infrequently, it may be hard to detect a day (or weather) effect using small samples unless the effect is really strong. It doesn't appear to be strong, if it exists.

I won't replicate the data sets here but I will provide the results of "regressing" hit rate on level. This is similar to a test for two binomial proportions, except that there is an assumption of of a linear relationship (correlation) between the response proportion and the predictor variable.

Experimental conditions: 82 INT, 230 elemental magic skill.

             Analysis Of Parameter Estimates

                     Standard  Wald 95% Confidence    Chi-
Parameter  Estimate     Error        Limits         Square

Intercept    8.3549    1.5956   5.2277     11.4822   27.42
level       -0.1015    0.0211  -0.1429     -0.0601   23.09

I left in the chi-square statistics this time so you can compare to the results of a test for two proportions (which has a chi-square statistic of 22.0426 with 1 degree of freedom).

Conclusion: For Steelshells, magic hit rate appears to be reduced by .10 for level 76 Steelshells compared to level 75 Steelshells, but this point estimate is not that precise. That seems like a fairly big effect if we can generalize this result, considering the types of VT-IT monsters a black mage might fight and still have a low resist rate on with decent merits and equipment (thinking Aura Statues), but I don't see any reason to doubt these results.

Summary

There is one more thing I want to address but I've been plinking away at this far too long and so I'll revisit it tomorrow, perhaps.

Here are some tentative conclusions so far:

• Below .50 "base" magic hit rate:
  • HQ staves co-aligned in element with that of a direct-damage magic spell (and probably other types too) appear to provide a constant increase of .15 magic hit rate
  • NQ staves co-aligned in element with that of a direct-damage magic spell (and probably other types too) appear to provide a constant increase of .10 magic hit rate
  • 1 point of elemental magic skill (from equipment alone?) corresponds to an increase of about .005 magic hit rate
• Above .50 "base" magic hit rate:
  • HQ staves co-aligned in element with that of a direct-damage magic spell (and probably other types too) appear to provide a constant increase of .30 magic hit rate
  • NQ staves co-aligned in element with that of a direct-damage magic spell (and probably other types too) appear to provide a constant increase of .20 magic hit rate
  • 1 point of elemental magic skill (from equipment alone?) corresponds to an increase of about .01 magic hit rate
• A magic hit rate penalty appears to exist when targeting monsters higher level than you

"Open questions": what about INT and magic accuracy?

I made an attempt at proofreading this post, but if there are any mistakes or non-sensical comments you may specify them in the comments (as though I'll receive any comments though). Criticism and any other insights about the validity of the data, commentary, and statistical analysis are welcome, too.

A half-year in parses

December 11: I now have the time to add some comments for all the parser output I posted last week.

Treasure and Tribulations BCNM, 1st attempt (July 11)

Melee Damage
Player            Melee Dmg   Hit/Miss  M.Low/Hi    M.Avg
NIN/WAR                 470      38/85      4/18    11.62


Spell Damage
Player                 Spell Dmg   Spell %  #Spells  S.Low/Hi     S.Avg
NIN/WAR                      914   64.55 %       29      4/44     31.52
- Doton: Ni                  164   17.94 %        4     40/44     41.00
- Huton: Ni                  140   15.32 %        4     20/40     35.00
- Hyoton: Ni                 200   21.88 %        6     20/40     33.33
- Katon: Ni                  110   12.04 %        5     10/40     22.00
- Raiton: Ni                 196   21.44 %        5     36/40     39.20
- Suiton: Ni                 104   11.38 %        5      4/40     20.80

Comments: it certainly is more palatable to fight a mimic (Small Box) straight up rather than hope you pick the right treasure chest. Comments on FFXIclopedia recommend sushi "except if you have really good gear," but melee accuracy against this mimic was a joke. I felt better off using the "wheel" lest the fight take 25 minutes.

Treasure and Tribulations BCNM, 2nd attempt (July 12)

Melee Damage
Player            Melee Dmg   Hit/Miss  M.Low/Hi    M.Avg
NIN/WAR                 214      20/78      5/13     8.47


Spell Damage
Player                 Spell Dmg   Spell %  #Spells  S.Low/Hi     S.Avg
NIN/WAR                     1008   80.45 %       36      4/44     28.00
- Doton: Ni                  115   11.41 %        5      5/40     23.00
- Huton: Ni                  190   18.85 %        6     10/40     31.67
- Hyoton: Ni                 220   21.83 %        7     20/40     31.43
- Katon: Ni                  164   16.27 %        7      4/40     23.43
- Raiton: Ni                 145   14.38 %        6      5/40     24.17
- Suiton: Ni                 174   17.26 %        5     10/44     34.80

Comments: more of the same (Small Box again), mainly to corroborate the hideous evasion of these mimics. I am curious whether there is any difference in hit rate targeting the larger boxes instead.

Evasion vs. Water Leaper (August 1)

Attacks Against:
Player           Total   Avoided   Avoid %
NIN/THF            253       247   97.63 %


Standard Defenses
Player           M.Evade  M.Evade %   Shadow  Shadow %   Parry  Parry %
NIN/THF              148    58.73 %       93   93.94 %       6   5.77 %

Comments: I trot out the thief support job to maximize my evasion. (I've seen "Evasion Bonus II" job trait from thief to be both +22 and +23 total.) This may be indispensable for something like Fenrir (I may try soloing it again now that Reraise effects can't be dispelled) but for mundane things not so much. Trading 12 or 13 evasion for all the abilities available to DNC37 (dancer also gets an Evasion Bonus trait) seems like a no-brainer for menial tasks, if I can ever bother to finishing leveling it.

Evasion vs. Goblin Slaughtermen, Temenos - Northern Tower (August 8)

Attacks Against:
Player           Total   Avoided   Avoid %
NIN/THF            241       234   97.10 %


Standard Defenses
Player           M.Evade  M.Evade %   Shadow  Shadow %   Parry  Parry %
NIN/THF              155    65.13 %       71   91.03 %       8   9.64 %

Comments: Ninja soloing for AF+1 in Temenos seems "common" enough for those who have the patience and adequate equipment. I've tended to err toward mixing both haste and evasion if only to speed up the process just a little, so even without maximum evasion, one can still evade a fair amount of attacks. (At least I assume that was the case for this, one of my last Temenos runs.) Sadly, in the past I have actually timed out mainly because of mediocre DD output, but it doesn't really matter to me whether I finish in 20 minutes or 28 minutes.

Enfeebling Despot (October 10)

BLM/RDM
Debuff      # Times   # Successful   # No Effect   % Successful
Bind             26             21             0        80.77 %
Gravity           8              8             0       100.00 %
Poison II        12             11             0        91.67 %

RDM
Debuff      # Times   # Successful   # No Effect   % Successful
Bind              3              2             0        66.67 %
Gravity           6              6             0       100.00 %

Comments: I had such extraordinary success (by my standards) binding Despot that I feel this is an anomaly. I am pretty sure my enfeebling magic skill a few months ago was 269, which isn't good for BLM. Although binding isn't necessary for soloing Despot as a black mage (yes, I didn't solo it here), it can give you a little slack.

Pahluwan Khazagand effect on crit rate (October 16)

Melee Damage
Player            Hit/Miss   M.Avg  #Crit     Crit%
WAR/NIN             459/39  143.43     40    8.71 %
SAM/WAR (Askar)    581/184  140.84     54    9.29 %
MNK72/WAR36       1270/283   52.96    148   11.65 %

Total Experience : 19012
Number of Fights : 100
Start Time       : 10:06:51 AM
End Time         : 11:07:50 AM
Party Duration   : 1:00:58
Total Fight Time : 1:35:08
Avg Time/Fight   : 36.59 seconds
Avg Fight Length : 57.08 seconds
XP/Fight         : 190.12
XP/Minute        : 311.77
XP/Hour          : 18706.50

Comments: I am no fan of the "Mamool Ja north" merit camp, but whatever it takes. I even included the experience summary to show that the exp rate was great (by my standards). Also, I noticed that the monk was wearing the Pahluwan body piece. I have seen bandied about the claim that the crit bonus on Pahluwan is "broken," and I believe this nonsense originated from this idiotic post from 2006. Such fuckers don't realize that the margin of error associated with the sample crit rate in question, even for 718 total hits, will be fairly wide. For example, a 95% Clopper-Pearson interval for the crit rate with Pahluwan body is (0.1366717, 0.1920368), so I wouldn't be talking shit about how the body makes the crit rate "worse."

Going back to the parser output, it seems to confirm the notion that critical hit rate is minimized at 5% (9% with 4 merits). This is to be expected without a sufficient amount of dexterity at this camp. If the monk's base crit rate before equipment was indeed 9% (assuming the monk had all the merits), then there is strong evidence that Pahluwan body does have an effect (trivial conclusion since the item description explicitly states there is one). As for the magnitude of the effect, a 95% CI for the crit rate bonus is (0.00939805, 0.04546965), so I am 95% confident that the true bonus is somewhere in that interval. So much for crit rate being "broken" (not that the effect isn't weak).

Enfeebling Aura Statues

≥ 82 (October 23)

BLM/RDM
Debuff      # Times   # Successful   # No Effect   % Successful
Bind             92             52             0        56.52 %
Dispel            1              0             1         0.00 %
Gravity         234            184             1        78.63 %
Sleep            34             22             0        64.71 %
Sleep II        120             94             0        78.33 %
Sleepga II        1              1             0       100.00 %
Stun             40             39             0        97.50 %

≥ 25 (October 24)

BLM/RDM
Debuff      # Times   # Successful   # No Effect   % Successful
Bind             11              8             0        72.73 %
Gravity          80             62             0        77.50 %
Sleep            11              7             0        63.64 %
Sleep II         26             24             0        92.31 %
Stun             22             22             0       100.00 %

≥ 9 (November 13)

BLM/RDM
Debuff      # Times   # Successful   # No Effect   % Successful
Bind              6              4             0        66.67 %
Gravity          25             20             0        80.00 %
Sleep II          5              5             0       100.00 %
Stun              8              8             0       100.00 %

Comments: Now that I have some working hypothesis on the relationship between magic skill and magic "hit rate" (again, to make a distinction between a "lack of resist" rate and the magic accuracy attribute), I am going to put it to the test against Aura Statues once I reach 289 enfeebling magic skill. (Merciful Cape is absolutely out of the question as I am not that masochistic; Enfeebling Torque is overpriced and obtaining Wizard's Coat +1 is contingent on luck getting the materials.) Oddly, the resist rate estimates seem consistent only for gravity. I'll have to look into it. Again, I wouldn't be surprised that a level correction plays some role.

Direct magic damage to Genbu (October 26)

Bio II
    3:   17
   35:    1
Burst II
 1067:    1
Thundaga III
  532:    1
Thunder IV
   73:    1
   99:    3
  199:    1
  398:    3
  795:    1
  798:   12

Comments: I seemed to have pretty good success damaging Genbu this time.

Dancer (lv14-15) EXP/hour (November 7)

Total Experience : 5845
Number of Fights : 82
Start Time       : 3:07:05 PM
End Time         : 4:17:42 PM
Party Duration   : 1:10:37
Total Fight Time : 1:43:29
Avg Time/Fight   : 51.68 seconds
Avg Fight Length : 75.73 seconds
XP/Fight         : 71.28
XP/Minute        : 82.76
XP/Hour          : 4965.81

Mob Listing
Mob                        Base XP   Number   Avg Fight Time
Akbaba                         ---        1             0.00
Canyon Crawler                  80        1            35.00
Canyon Rarab                    60        2            24.50
Canyon Rarab                    65        5            29.01
Canyon Rarab                    70        9            40.56
Canyon Rarab                    75        4            49.29
Goblin Digger                   80        1          1:33.09
Goblin Thug                     60        1         37:07.29
Goblin Thug                     65        1            35.00
Goblin Tinkerer                 80        1            54.01
Goblin Tinkerer                 90        1          1:00.04
Killer Bee                      70        6            36.41
Killer Bee                      75        5            38.22
Killer Bee                      80        4          2:37.06
Pygmaioi                        65        3            34.68
Pygmaioi                        70        3            50.68
Pygmaioi                        75        7            45.59
Pygmaioi                        80        2          3:35.61
Strolling Sapling               65        8            33.02
Strolling Sapling               70       10            42.63
Strolling Sapling               75        1             6.00
Yagudo Acolyte                  60        2            19.51
Yagudo Persecutor               90        2            42.54
Yagudo Piper                    90        1          1:01.01
Yagudo Scribe                   60        1            13.01
Yagudo Scribe                   65        1            10.00

Comments: I've become progressively less patient with leveling subjobs even though the last few have been easy to solo (against goblin pets), from samurai to dark knight to red mage and, now, dancer. I just don't see myself leveling another job as my playing time wanes, especially considering no job other than dancer will let me spend 70 minutes mowing down every EP nonstop for almost 5k exp/hr.

On sophistry

(Edit - Dec. 23: changed link to document.)

Two years too late, but I prepared some comments on this so-called "advanced analysis" of paralyze proc data, mainly concerning the statistical sophistry involved. (I really hope insights have been further developed since then.) Such are the perils of idleness. (I don't recommend that you continue reading further; you've been warned.) I address specific sections of the write-up (sections in boldface).

Introduction

The author claims that it is not desirable to maximize the duration of a paralyze effect. Instead, he (is it ever a she when bloviating about some B.S.?) seems to think that maximizing the number of processes (procs) per cast is the relevant goal. He cites two hypothetical situations where the durations are different yet the rate of procs per unit time is the same. He argues that the scenario with the shorter duration gives an opportunity to reapply a possibly stronger paralyze (higher rate of procs per unit time).

However, he proposed a model that assumes that MND, enfeebling magic skill, and a HQ staff have an effect (statistically significant or not) on both spell duration and the number of paralyze procs. So why not just model the rate of procs per unit time to begin with? The author argues we must "account for" (control for) the effect of duration (something we cannot directly control) so we can see how the controlled factors affect the number of procs directly within some varying time interval that is supposed to be under statistical control. But this is also modeling the rate of procs per unit time (when duration is controlled).

Finally, his "analysis" shows that the duration of the paralyze effect has the greatest effect on the number of procs (MND also does), which he considers unfortunate. However, it goes without saying (but I'll say it anyway) that you cannot change duration purposefully without changing some combination of MND and enfeebling skill (not to mention any omitted variables that may affect duration). (In most practical situations MP-users don't cast without elemental staves.) So what, exactly, did you expect?

Preliminary Analysis

Note that the presence of the 10 missing observations affects the calculation of the correlation matrix. The missing observations are excluded from the subsequent path analysis.


Path Analysis

First off, I must acknowledge that I have never used path analysis for anything, so as I become more familiar with it I may revise my comments later.

The pair-wise "sample" correlations between the so-called exogenous variables here, MND, enfeebling skill, and HQ staff, are meaningless as the variables are not random. (What multicollinearity?) I don't even know why they are indicated on the diagram other than to follow some rote procedure rigidly.

"Clustered ordinary least-squares (OLS) regression" is an oxymoron. Generally speaking, using a robust least-squares method of estimation is a departure from what is ordinarily done. Furthermore, the justification for "clustered robust" LS estimation--that observations within each group (naked, enfeebling, MND, etc.) are not independent--is not valid. The author attributes lack of independence of observations within groups to the "experimental setup of this test," but there is absolutely nothing in the description of the "experimental setup" that suggests this should be so. Autocorrelation is not an issue. (Why would catoblepas build up resistance to paralyze anyway?) But even if it were, a "clustered robust" method cannot account for that. What he basically did was control for group effects twice, which is absolute nonsense and has no effect on his parameter point estimates anyway. (The coefficient of determination, R², is the same whether improperly accounting for nonexistent "clustering" or not.)

There is also the issue of not controlling for test subject (monster), but regardless of the magnitude of the effect of test subject, this concern is not discussed while comparatively more frivolous concerns are. To wit, the author's irrelevant aside about Bayesian inference has nothing to do with the use of BIC here, even though he is not really doing model selection but providing cover for arguing that MND may be a more "important" predictor of duration than the use of a HQ staff.

That cover is rather weak though since individual (non-simultaneous) interval estimates for the "standardized coefficients" are rather wide in the model that the author actually "chose":

MND: (.086, .404)
skill: (.017, .337)
staff: (.057, .377)

Now consider the second regression (modeling number of procs). Again, the author uses completely inappropriate clustered robust linear regression, which leads him to trump up enfeebling skill as highly significant. In reality, the enfeebling effect is barely significant at the 5% level, hardly convincing evidence of a real effect (if it exists, which I doubt). Moreover, something fishy could be going on with the last set of observations. If you omit those from the analysis, the enfeebling effect does not even approach significance. But the data are what they are.

Discussion

Again, the author fails to recognize the imprecision of his parameter estimates (standardized beta coefficients) despite curiously devoting time earlier to a frivolous comparison of two population correlations in Appendix A.

Today, it may be "commonly known" that MND does affect the accuracy of a MND-based magic spell in some way, but arguing that MND has a relatively stronger effect on paralyze duration (a measure of accuracy) than enfeebling skill on the basis of standardized effects is spurious because of the poor parameter estimates and because of the interpretation. Obviously, the main effects are not random variables, so their associated standard deviations don't have any particular meaning as they are just an artifact of experimental control.

Consider the interpretations in real-unit terms. From the first linear regression, the duration is estimated to increase by 6.38 seconds for every 22.8-point increase in MND (controlling for the other main effects). Similarly, the duration is estimated to increase by 4.93 seconds for every 14.4-point increase in enfeebling magic skill (controlling for the other main effects). Point for point, enfeebling magic skill is more effective than MND, and I don't know anyone who would argue for a comparison other than by a per-point basis.

Certainly, there are distinct levels of resists, but there is no reason to believe that HQ staves have a privileged role in determining the distribution of partial resists any more than other factors that affect magic "hit rate," especially since magic accuracy bonuses for both NQ and HQ staves have been estimated.

As for unexplained variability in the number of procs, the author provides a laundry list of possible explanatory factors, none of which are as important as the ones under one's direct control. (Do you do anything only during specific moon phases?)

Reaction and criticism (not in the write-up)

These people had the temerity to broadcast this "analysis" on both Allakhazam and Killing Ifrit.

On Allakhazam, you typically had the usual sucking off. Not unexpectedly, a reasonable objection was raised about the relationship between duration and number of procs. It seems practical enough to consider an increase in duration (holding other factors constant) as increasing the number of procs that are observed. The exogenous factors (MND), on the other hand, actually affect the potency of paralyze (proc rate), also measured as the number of procs, but holding duration constant. But instead of recognizing this line of reasoning, these numbnuts hid beind numbers (and statistics) without even thinking about how to interpret effects and the implications of their "analysis." (This is actually all too common for all the so-called "mathematicians" on Allakhazam.)

On Killing Ifrit, there were a few somewhat naïve criticisms of the experimental design (all from the same poster). Yes, it would be nice to use more than two levels of each independent variable, but there is no compelling case for a nonlinear trend. Again, generating standardized effects for each predictor is a pointless exercise for this data (as discussed previously). A multi-factor ANOVA is superfluous as you can construct simultaneous confidence intervals for the parameter estimates from regression (in general). Sample size and power are brought up, but concern for "too much power" (with excessive sample sizes) is simply a trivial objection.

Alternative (not in the write-up)

I don't have any particular objection to path analysis per se. The low-hanging fruit are that the statistical procedures are questionable, the write-up mired in irrelevant details and the interpretations awkward.

Let us return to the original motivation for the path "analysis." Modeling proc rate was criticized (false distinction between that and number of procs when controlling for duration) but the interpretations involved in path analysis concern proc rate anyway. (Potency must be a proc rate. This is beyond dispute.) So why not model the proc rate directly? (And if you care so much about modeling duration too, you can regress that on your favorite predictors. No one's stopping you.)

It seems natural enough to use Poisson regression to model proc rate, and I carried out this procedure in R (output below):

Call:
glm(formula = proc ~ MND + enfeebling + staff + iceday + offset(log(duration)), 
    family = poisson, data = paralyze)

Deviance Residuals: 
    Min       1Q   Median       3Q      Max  
-2.2293  -0.8721  -0.0776   0.6353   3.0779  

Coefficients:
             Estimate Std. Error z value Pr(>|z|)    
(Intercept) -6.260517   1.120601  -5.587 2.31e-08 ***
MND          0.007941   0.002259   3.516 0.000439 ***
enfeebling   0.008038   0.003644   2.206 0.027404 *  
staff        0.047913   0.107654   0.445 0.656271    
iceday      -0.027394   0.114415  -0.239 0.810772    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 

(Dispersion parameter for poisson family taken to be 1)

    Null deviance: 155.82  on 139  degrees of freedom
Residual deviance: 138.19  on 135  degrees of freedom
AIC: 510.53

Number of Fisher Scoring iterations: 5

The model deviance indicates that this model is an acceptable fit to the data. (Note: I facetiously specified an Iceday effect in the model.) Controlling for other factors, proc rate is estimated to increase by .797% for every one-point increase in MND. Note that the z-values are similar to the t-values using OLS estimation.

Crystal Stakes collapse

Note: this is a profanity-laced rant.

On November 4, the day of the last server maintenance, my winning rate in C1 races was 32/68. Since then, the results have been atrocious: 5 1st place finishes, 8 2nd or 3rd place finishes, and 3 below 3rd place. For perspective, in the first 68 races, I placed worse than 3rd only once. Or, to put it in terms that really make me incensed as I write this, a loss of chocobucks between 542 and 764 in sixteen fucking races, or 33.9 to 47.8 per fucking race, or between 72 and 103 fucking minutes wasted per race farming chocobucks for basically jack shit.

It's not merely that I have been losing but that losing more often entails dealing more frequently with a timesink that is in place basically to deter RMT. But so be it... not that I don't have methods of dealing with it.

During this period, aside from one uncontested race (and I didn't even finish first), all the C1 races I've entered (and recorded the toteboards for) have involved at least one other PC chocobo. Rationally, I must acknowledge that C1 races are more hotly contested than ever. Irrationally, I am pissed off that the same chocobos keep placing first over me (and to rub more salt in the wound, I end up placing behind garbage SS/SS/B/F chocobos and "off the podium"), even ones with nominally the same attribute profile as mine. Literally, the same SS/B/B/B chocobo has placed first 3 times against me while I have gotten 5 first-place finishes in three weeks. (Interestingly, I've observed that chocobo was raised with an enlarged beak, for what it's worth. I might even laugh if that owner read this blog to get some ideas.) It would be even more crackpot and solipsistic to associate this string of poor results with the latest server maintenance, but you can't count the FFXI "dev team" out for fucking with its players without even being upfront about it, especially that fat-fuck CoP director.

Sure, in the long run things may even out all things being equal (all things being equal is a huge assumption, not knowing what saddles they are using), if I even get a chance to even things out. But this is FFXI the zero-sum MMORPG, where illiterate, proudly ignorant, gloating motherfuckers get to rake it in and bolt once they get theirs (fuck the rest!) while you get jack shit for the same amount of "effort." Even in chocobo racing.

Estimating changes in magic hit rate with skill

For the purposes of estimating melee hit rate, the functional relationship among accuracy, dexterity, combat skill, mob level, and mob evasion has long been established, thanks to the clever use of the check function. Sadly, no such relationship has really been justified for magic "hit rate" (or resist rate), but that doesn't mean we are condemned to flail in the dark.

Having wondered myself about the utility of meriting elemental magic skill for the purposes of reducing the frequency of resists on "hard stuff," I looked for some information on the relationship between magic skill and resist rate, but solid evidence was hard to come by. Fortunately, after wading through senseless conjecture on BG, I managed to come across an interesting data set for which the "success" rates of casting magic on Ebony Puddings were recorded, given specific levels of elemental skill, magic accuracy, and INT. Even better, this data all but invites me to take a swing at it using some kind of linear regression analysis.

But first, if the factors that go into "magic hit rate" (rate of success or rate of no resists) are similar to those that go into melee hit rate, there are several issues that immediately come to mind when trying to suss out some kind of relationship, such as the relationship between magic accuracy and magic resistance (or evasion?). (Dec. 15: I wrote "Is a ratio involved, as is the case with melee accuracy and melee evasion?" which is incorrect. I probably was thinking of MAB/MDB, but that would be analogous to melee attack and defense.) Furthermore, even if magic resistance/evasion were constant among the flans on Mount Zhayolm, there is a range of levels for Ebony Puddings (supposedly 75-80 on Mount Zhayolm), and if a "magic hit rate" calculation involves a level correction, there is no practical way to account for that.

Still, looking specifically at the nuke data (tests II, III, IV), there appears to be some evidence of a linear association between magic skill alone (holding other relevant factors constant) and success rate. You can do your own plot if you're not convinced.

But as far as magic accuracy is concerned, there are only three combinations of magic accuracy and elemental skill where the success rate was measured. One may argue that magic accuracy seems to be less effective at 242 elemental skill than at higher levels of skill, which may seem persuasive (random variability and unaccounted sources of variability notwithstanding). Really, though, it's a reach to conclude that elemental skill and magic accuracy are correlated with the limited data here.

Finally, INT seems to have no effect at 242 elemental skill, yet has some effect in large quantities at 274 skill. Maybe it's not all that far-fetched to say that the effect of INT on magic hit rate is dependent on magic skill level, which can compromise the estimates associated with a regression analysis. Even worse, perhaps the relationship between INT and magic hit rate (holding other factors constant) is not strictly linear but follows some weird piecewise function depending on your target mob's INT. This calls attention to the need for more data at other levels of INT, macc, and elemental skill (or perhaps a better choice of target whose level and magic resistance value is known to be fixed, but in practice this will be extremely difficult to achieve).

At any rate, using linear regression (with unresisted magic hit rate as the binary response) on the above observations (ignoring the middle rows of test II because they contribute to a poor model fit) gives the following parameter estimates (I truncated output to save space):

                     Standard   Wald 95% Confidence
Parameter  Estimate     Error         Limits         Pr > ChiSq

Intercept   -1.9393    0.1872   -2.3062     -1.5724      <.0001
skill        0.0095    0.0007    0.0082      0.0109      <.0001
macc         0.0147    0.0022    0.0103      0.0190      <.0001
int          0.0028    0.0010    0.0009      0.0047      0.0038

I included both INT and magic accuracy in the model just for the heck of it even though the parameter estimates associated with them aren't all that reliable. Certainly, including more observations with varying levels of INT and magic accuracy may improve those estimates (assuming magic hit rate is linear over some range of either factor), and they should be included in a model for the sake of a comprehensive view of magic hit rate. But for now, we can see that the data suggest that magic hit rate increases by about 1% for every one-point increase in elemental magic skill (holding INT and magic accuracy fixed). The range of elemental magic skill considered is between 242 and 295.

One can also perform a similar analysis with the Sleep trials (tests V and VI), but note that the "success" rate encompasses partial resists also:

                     Standard   Wald 95% Confidence
Parameter  Estimate     Error         Limits         Pr > ChiSq

Intercept   -1.3636    0.5853   -2.5108     -0.2164      0.0198
skill        0.0056    0.0018    0.0021      0.0091      0.0016
macc         0.0085    0.0025    0.0035      0.0134      0.0008

It seems that the effects of magic skill (enfeebling in this case) and magic accuracy are weaker for sleeping than for nuking. (Granted, the interval estimates are rather wide.) The range of enfeebling magic skill is between 307 and 333. It's possible that the acts of sleeping and nuking are just not comparable (unlikely) with respect to resist rates. It's also possible that the effects of general magic skill and accuracy on magic hit rate are diminished past the 300 level of general magic skill. Either way, this complicates understanding of magic hit rate somewhat and steps can be taken to rule out either explanation.

It hasn't escaped my attention that magic accuracy seems to increase magic hit rate more than magic skill, ignoring the wide interval estimates. If this is really the case, the difference is so slight and direct competition between the two attributes so rare that it's not worth caring about. Even comparing Oracle's Robe (magic accuracy +6) to Igqira Weskit (elemental magic skill +5), I would first argue the benefits of using Oracle's Robe to replace both Errant Houppelande (like anyone cares about the elemental enfeebling line) and Igqira Weskit. The HP+20 for Sorcerer's Ring activation can be useful, too.

It also occurred to me that one may try to argue, in analogy to melee accuracy and melee hit rate, that this data support the contention that magic skill increases magic hit rate by 0.9% above the 200 skill level (1% at or below 200), although it is ludicrous to distinguish between 0.9% and 1% based on random data without excessive sample sizes.

But, if all you cared about was estimating the change in magic hit rate for every one-point increase in elemental skill, you might as well focus on the change in magic hit rate between two levels of elemental skill that are relatively far apart, assuming the rate of change is constant (in other words, a linear relationship between hit rate and skill), an assumption that is borne out by the previously considered data.

The regression analysis for the nuke data used 1,400 total trials; these trials could be allocated equally between, say, 242 skill and 292 skill. Then you'll have an easier time showing that the increase in magic hit rate is less than 50% (less than 1% per point of elemental skill). (Use a test for two proportions.)

Aggressor and double attack merits

After meriting on greater colibri for a bit, I was wondering whether I would be "better off" had I merited double attack to level 5 instead of Aggressor recast. (Unsynchronized Berserk and Aggressor timers would be really annoying though.) This May 2007 discussion comparing Aggressor and double attack merits shows, despite the muddled presentation, a situation where fully merited double attack is more effective than fully merited Aggressor recast, since Aggressor supposedly provides an accuracy bonus of 25, which corresponds to only a 12.5% hit rate increase (on average). However, we might be interested in the magnitude of difference between the two Group 1 schemes, which is more difficult to quantify.

One approach is to calculate the average number of attack rounds to reach 100 TP for both 5 DA/0 Aggressor and 0 DA/5 Aggressor. (The number of attack rounds is independent of specific damage values.) Of course, the relative effectiveness of Aggressor is higher when your hit rate is lower, as is usually the case when targeting anything more difficult than greater colibri. Then it might be useful to compare max DA and max Aggressor for lower levels of a baseline hit rate.

Ultimately we want to know what the differences in long-run "damage over time" are, but first we can look at the average number of attack rounds, as that is an indirect measure of time. (Assume number of seconds per attack round is constant.) Unfortunately, an analytic expression of the average number of attack rounds to reach 100 TP is too annoying to derive primarily because the number of attack rounds needed to reach 100 TP depends on the TP return of the previous weapon skill, which is almost never zero for a multi-hit weapon skill with a decent hit rate. The number of hits to 100, given initial TP, seems basically to follow a Poisson process, but I'd rather not worry about cumbersome calculations. Therefore, I resorted to simulation to generate the following approximate values based on my warrior setup (varying the Group 1 merit configurations, obviously), given baseline hit rate and the use of a 3-hit weapon skill (Raging Rush or King's Justice):

Average number of attack rounds given baseline hit rate

       5/0   2/4   0/5
0.2   20.19 19.81 19.87
0.3   14.60 14.52 14.61
0.4   11.40 11.39 11.47
0.5    9.31  9.33  9.41
0.6    7.83  7.86  7.95
0.7    6.73  6.78  6.84
0.75   6.29  6.33  6.39
0.8    5.88  5.93  5.99
0.825  5.71  5.74  5.81

Here, the first column corresponds to baseline hit rate (before the Aggressor bonus), and the next three columns correspond to different Group 1 merit configurations:

"5/0": 5 double attack, 0 Aggressor
"2/4": 2 double attack, 4 Aggressor (mine)
"0/5": 0 double attack, 5 Aggressor

Then, we can obtain values representing "damage over time" in terms of hits per round, given the baseline (or nominal) hit rate:

Average number of hits per round given baseline hit rate

       5/0   2/4   0/5
0.2   0.336 0.341 0.340
0.3   0.458 0.460 0.456
0.4   0.580 0.579 0.573
0.5   0.702 0.698 0.691
0.6   0.819 0.818 0.811
0.7   0.946 0.935 0.925
0.75  1.006 0.996 0.983
0.8   1.069 1.054 1.041
0.825 1.097 1.085 1.071

The max DA configuration is already about even with max Aggressor at 30% baseline hit rate, and it really starts to pull away as the baseline hit rate increases (especially after the point where Aggressor does not provide the full accuracy bonus, past 82.5% hit rate), so to me there is scant justification for 5/5 Aggressor. This makes sense as fully merited Aggressor provides an average 1.5% hit rate increase over non-merited Aggressor, which pales in comparison to the increase in "damage over time" that can be conferred by 5 double attack in the presence of high levels of accuracy. This analysis doesn't account for multi-hit weapons such as Ridill and Joyeuse, but the relative differences between 5/0 and 0/5 should still favor 5 DA merits even though the gap may close. And of course, this post doesn't account for actual damage per hit, but DA and hit rate are "independent" of damage per hit anyway (hits/time × damage/hit = damage/time!) and it's not that much of a reach to estimate real "damage over time" by factoring in an average damage per hit.

I found it helpful to plot attack rounds vs. hit rate to illustrate that the average number of attack rounds to 100 TP levels off as hit rate increases:



Obviously the rate of change in the number of attack rounds to 100 TP is decreasing in magnitude (but is still negative) with hit rate. But the number of attack rounds is not a direct measure of damage over time. Damage over time is a ratio of, yes, damage over time. The number of attack rounds is a proxy for time, and is not a ratio.

The number of hits, given the number of attack rounds, on the other hand, is a measure of damage, so dividing the number of hits by the number of attack rounds gives a quantity that can stand in for "damage over time," as plotted below vs. nominal hit rate:



Of course, there is no reason to plot such a thing because intuitively the rate of change of hits/round must be constant (we're plotting hit rate vs. hit rate!), especially if you believe that 2 points of accuracy always corresponds to 1% hit rate between 20% hit rate and 95% hit rate. If you do, it's complete nonsense to speak of damage over time showing "diminishing returns" to hit rate. Hit rate leveling off with accuracy in some logistic fashion is another story though.