Can BABS really be used during the season?

In the recent readership survey, I received no fewer than a dozen comments about using BABS during the season. “How do I use it? How can it help me? Is it even a viable tool?”

These are difficult questions.

We want to be able to use BABS during the season. Most of you have bought into the philosophy behind BABS and the conceptual approach she uses on Draft Day. But does the same modus operandi work during the season?

There is a chapter in “The BABS Project” ebook that talks about using BABS in-season. I encourage you to read it again; it contains some helpful tips. But there is also an underlying harsh reality that has remained unspoken.

Simply, BABS is a long-view tool. I spent a good part of the early chapters of the book talking about the dangers of small sample sizes, recency bias and the Fear of Missing Out. Any type of in-season analysis flies in direct opposition to all of those. Everything we are trying to analyze now – from validating Eric Thames’ batting heroics to trying to discount Danny Salazar (pictured) and his 5-plus ERA – is a study in small samples, recent performance and the fear of waiting too long to do something.

These challenges are not new. We have always faced these obstacles in our previous analyses. The difference between old-style and BABS-style small-sample analysis lies in our perception of accuracy inherent in the faux-precision of the statistics.

I can dig into Danny Salazar’s support metrics and see that his 5.66 ERA obscures a 3.18 xFIP, telling me that he still possesses exemplary skills. We then take the leap (not wholly unfounded) that his “luck” will change and his surface stats should improve. Given the nearly two-and-a-half run variance between ERA and xFIP, some might even conclude that he should see significant improvement.

Here is Salazar’s BABS scan:

Year   Team    IP     W       K      ERA     WHIP   PT    Er     K
====   ====   ===     ==     ===     ====    ====   ==    ==    ==
2014   CLE    110      6     120     4.25    1.38    M          KK
2015   CLE    185     14     195     3.45    1.13    F     e    KK
2016   CLE    137     11     161     3.87    1.34    M          K+
2017   CLE     41      2      62     5.66    1.57    M          K+
Proj   CLE                                           F     e    KK

There is nothing here that blatantly shouts about an ERA improvement. In fact, BABS has never been all that impressed with Salazar’s ability to put up decent ERAs. We might expect some natural regression to the mean here – a 5.66 ERA does seem like it’s an outlier – but BABS isn’t jumping up and down yelling “grab him now!” like an ERA-to-xFIP analysis might.

Which is correct? Given that we’re still talking about a small sample of innings, I don’t know. A missing piece of the BABS profile above is Salazar’s injury Liability (inj-), which essentially says that any expectation is going to face a larger than expected error bar. Perhaps that is what we are seeing now.

But it’s all inexact, and as fantasy league managers, fuzzy greyness is the bane of our existence. We’d much rather be able to hang our hat on an ERA-to-xFIP variance so we can make a decision, even if that analysis is not any more accurate than BABS, because of sample size.

I know that’s not what you want to hear, but that’s the reality. Beyond the specific tactical tricks described in the ebook, this is an inherent limitation to BABS. Still, I am not about to wash my hands of the whole thing. BABS is a living, breathing, evolving creature and I will continue to fine tune the ratings. Perhaps further work will reveal some elements that are more projectable with smaller sample sizes.

For now, let me address the specific comments in the surveys:

Is BABS valuable as an in-season tool based on short-term performance?
With limitations. It was not inherently designed for short-term performances.

Can BABS or the underlying analytics tell me if dramatic changes in Assets or Liabilities are real?
Sometimes. If a player shows a dramatic change and the ratings show little or no change, that is a good indication that a surface change is not real. Conversely, if a player is performing to expectation but his BABS rating shows significant change, take note. Small sample warnings still apply in all cases.

There needs to be a way to easily see who the best free agents are in a specific league once the season starts and transactions happen every day.
You can always look up free agents in the BABS database and analyze their scans. Minor league call-ups are problematic, as described in the ebook. Small sample warnings still apply in all cases.

I am still working with figuring out the best way to compare the pre-season BABS with in-season BABS for a more efficient way to look for buy-low (or sell high) opportunities.
I would look less at the pre-season BABS and more at the multi-year scans in the database. They describe who the player is based on what he has actually done, not based on what BABS thought he might do. Using the Salazar scan above, BABS shows that his underlying skills right now are no different from what he posted last year. That is important information. However, small sample warnings still apply.

It would be valuable if I knew how to use BABS to identify which hot starts to buy and which to sell.
As noted above, it’s tough to use BABS’ long-view perspective for short-view analysis.

Jury is out whether BABS is more than a draft prep tool.
It is entirely possible that this is where we will end up. As much as we want to try to force a round peg into a square hole, we might just be better off making sure BABS is completely optimized for draft application. I am not at this point yet, but I have not ruled it out.