Kill Rates and Survivorship Bias
No not bias towards survivors. Survivorship Bias.
https://en.m.wikipedia.org/wiki/Survivorship_bias
In short terms, survivorship bias is a statistical bias when interpreting data, where only certain data is collected because other sources don't "survive" to be collected. Now onto the meat of it.
Dead by Daylight devs put a strong focus on balancing around kill rates. This makes sense on the surface, but anyone who plays will tell you some killers are better at pub vs swf, or high skill vs low skill. That makes kill rates a dubious metric.
But it gets a lot worse when you consider how MMR is calculated. MMR is calculated on a per killer basis. This means that mmr levels per killer will fluctuate wildly even for a single player. Furthermore, as a player moves up in mmr on a particular killer, they may find themselves hitting a wall or getting frustrated. They naturally switch to a better killer. This creates an MMR bias where weaker killers will be much less likely to REACH high mmr.
Sure you have your absolute legends who take a Trapper or Pig to high mmr, but this requires they play at such a high level that they are effectively much better than players in their bracket. All of this bias means kill rates are even less meaningful than they would be in other multiplayer games. Because the survivorship bias means that bad killers will be more common in low mmr, even for a sample size of high mmr killer players!
All of this together paints an atrocious picture. Kill rates are always questionable as a metric in any game. But because DBD locks each killer in a separate box even for the same player, kill rates becomes completely meaningless. I have to question if anyone at Behavior is aware of this, or even the community at large.
The compartmentalization of killer MMR destroys any attempt at data collection, because skill levels are not standardized.
Comments
-
It is my understanding that while individual killers have distinct MMR rankings (with only slight variance between), a single player regardless of how much or little they play specific killers will have an overall MMR ranking. So if I have like 1500 on a Hag then start playing Cannibal, I don’t start at 0 MMR on cannibal; he is more likely around 1200. The devs do this because while different killers have different powers and preferable play styles, the core mechanic is the same for each of them. If I’m experienced enough to be at high MMR with one killer, picking up any other killer and rocking with him is significantly easier than if I’m inexperienced with all killers.
5 -
This is correct for killer MMR. It's not a straight average of them all, but a veteran killer will always have a decent starting number because they still know the basics even when first playing a new killer.
3 -
But we don't have the specifics. And personally I do notice pretty big skill differences between survivors when I switch killers. Given the number of balance and bug issues in the game's history, I can see the rubber banding causing an issue with how they analyze kill rates.
0 -
It's so funny to me, that Pig got "buffed" and people still see her as bottom tier.
Guess she needs to be on a watch list right?
0 -
Assuming the matchmaking is allowed to function (no backfills, sufficient number of players available for that MMR grouping) then there is a range you get the first couple of trials when running a new killer.
But you're right we don't have enough info to know exactly how it works. I think the primary intent is to prevent veteran players from mixing with newbies and/or very casual players. Over time it will even out, but like I said their matchmaking system needs to be allowed to function consistently first and foremost.
0 -
Survivorship bias: Of all the forms of selection bias that exist, not sure why you went with this.
But that's not hugely important, people talk about selective remembering of data pretty frequently. I've seen BHVR discuss it as well.
Dead by Daylight devs put a strong focus on balancing around kill rates.I wouldn't say a strong focus and I think this is a common mistake people make. Every time I've seen BHVR post about this they've discussed having multiple factors they consider in balancing a killer.
This makes sense on the surface, but anyone who plays will tell you some killers are better at pub vs swf, or high skill vs low skill.BHVR would probably tell you this as well. It's a presumption that BHVR doesn't want the killers to be unbalanced. They seem absolutely okay with some killers being starting killers, and others being the killers you play once you get good at the game.
Kill rates are always questionable as a metric in any game. But because DBD locks each killer in a separate box even for the same player, kill rates becomes completely meaningless.If you assumed that was all they were going off of, then yes. However BHVR's internal data almost certainly allows them to break the killer performances down by different MMR brackets.
2 -
Buffed pig is still not that good, while yes pub players will go down to no-blink nurse, but it doesnt make that a good killer no?
Like anything pig can do singularity, blight, spirit, oni, nurse, unknown and the other good killers can do better. While pig is no freddy she is still outclassed by better killers.
0 -
These are some hefty assumptions without any data reference.
" This creates an MMR bias where weaker killers will be much less likely to REACH high mmr."
There has never been, at least to my knowledge, any data released about killer distribution over MMR. We don't see MMR either. These are assumptions stacked on assumptions
4 -
Are you also factoring in those who don't take games seriously, which wind up as Survivor side, take a team fall to say "thank you killer for being fun and not sweaty, we who are about to die, salute you" and or these killer's who just do Rifts and Dailies and if they don't require killing, let off survivors as long as the challenges are met?... or those who truly tunnel out all one by one? Just for the 4k while going against survivors that just give up allowing their demise?
1 -
I'm not sure "weaker killers will have a lower average MMR" is that outrageous of an assumption, but your point about BHVR not being transparent enough with their stats is good
1 -
I didn't say anything else, did i?
I wanted to point out how hilarious it is to me, that while there are a lot of killers that are stronger, which you pointed out, the Pig has to be on a watch list.
0 -
My bad then, misunderstood the original comment.
1 -
All good!
Have a 👉🐽
0 -
I mean this is so obvious, but thanks for pointing it out to everyone else. There's been a lot of citing the kill rates to justify killer nerfs these past few years, so I guess people still don't get this. There's a clear disconnect between the stats and what you see when you play.
0 -
There's definitely skill differences. No one was claiming there wasn't. The initial MMR ranking of a killer you just started on will be fairly close to your average killer MMR though rather than assume you are brand new to the game. That MMR will then settle to it's rightful place depending on how well you do. I'm not very good at nurse, so if I decide to play nurse... I'll be faced with drastically worse survivors than if I played Dredge. That doesnt mean that I'd be thrashing those worse survivors though since... I suck at nurse, lol.
0
