Kelly Criterion Derivation (math heavy)

[callipygian]

As I was thinking about replenishable bankrolls, the idea of re-deriving the Kelly Criterion came up. I looked into some derivations of the simple Kelly Criterion, but I'm a little confused.

http://www.jimgeary.com/poker/letters/KELLY.HTM (Archive copy)

The line I have the most trouble with is:

B' = [(1+v*f)^w]*[(1-f)^(n-w)]*B

B' = new bankroll after n plays
B = original bankroll
v = payout on win
f = Kelly fraction
n = hands played
w = hands won

There are two things that are objectionable to me.

(1) There's no inclusion of probability of pushing; for games such as Pai Gow Poker where pushes are very common this would certainly factor into bet sizing. However, this seems easy to compensate for. Later in the text when p = w/n, you'd just introduce p = w/(n+x), where x is the probability of pushing and n is the total hands won or lost.

(2) This doesn't seem to weigh the statistical probability that hands are more likely to be WWLL than WWWW. That is, if n = 2 and the payout even, the probability that you have a win and a loss (and are basically even) are weighed equally with the probability of two wins.

It seems to me that the proper formula would be derived as such:

New bankroll = Old Bankroll * (Probability of win * Amount after Win + Probability of Loss * Amount After Loss)

B' = B*[p*(1+vf)+(1-p)*(1-f)]^n

which would then weigh the outcomes according to the probability that they appear.

Setting dB'/df = 0 gives f* = 1/[p(v+1)-1], which is not the otherwise-derived value of f* = [p(v+1)-1]/v.

Where did I go wrong?

 

[Sonny]

(1) There's no inclusion of probability of pushing; for games such as Pai Gow Poker where pushes are very common this would certainly factor into bet sizing. However, this seems easy to compensate for. Later in the text when p = w/n, you'd just introduce p = w/(n+x), where x is the probability of pushing and n is the total hands won or lost.

You could simplify it even more than that. Since a push doesn’t affect the size of your bankroll at all you can simply ignore it. The problem with that formula is that is uses (n-w), which assumes that all non-winning hands are losses. All you have to do is replace that with the probability of a loss and ignore the pushes. Winston Yamashita uses that simplification in his Kelly article:

Bn=[(1+f)^W][(1-f)^L]Bi

Bi= initial bankroll
Bn= bankroll after N bets
f= fraction of bankroll bet
N= number of bets
W= number of wins
L= number of loses

http://www.bjmath.com/bjmath/kelly/kelly1.htm (Archive copy)

However, his formula assumes even money payouts.

(2) This doesn't seem to weigh the statistical probability that hands are more likely to be WWLL than WWWW. That is, if n = 2 and the payout even, the probability that you have a win and a loss (and are basically even) are weighed equally with the probability of two wins.

They are only weighted equally if it is a fair game. Once the wins and losses start to diverge, the probabilities shift. It’s similar to how the binomial distribution works:

C(N,L)*p^W*q^L

N= Number of trials
W= Number of Wins
L= Number of losses
p= Probability of a Win
q= Probability of a Loss (not necessarily 1-p if ties are allowed)
C(N,L)= Combination of N events takes L at a time (N choose L)

-Sonny-

 

[callipygian]

Bn=[(1+f)^W][(1-f)^L]Bi

This still multiplies the two probabilities, which is what I have problems with.

C(N,L)*p^W*q^L

Expansion of the (a+b)^n term in my equations will give a binomial distribution.

(a+b)^n = Sum{C(n,i)*a^(n-i)*b(i)}

 

[Sonny]

This still multiplies the two probabilities, which is what I have problems with.

You have to multiply the probabilities in order to preserve the constant resizing of the bankroll. Let’s compare the two formulas by looking at a simple example. We’ll start with a $100 bankroll betting 1% on each hand we play. After playing two hands we have won once and lost once. Here are the results of the two formulas:

B = $100
f = 1%
v = 1 (even money payouts)
W = 1 win
L = 1 loss
N = W+L = 2
p = W/N = 50%
q = L/N = 50%

Calliphygian:
B’ = B*[p*(1+vf)+(1-p)*(1-f)]^n
B’ = $100*[0.5*1.01+0.5*0.99]^2
B’ = $100

As we can see, with your formula we have broken even. That is what we would expect after one win and one loss, right? They would cancel out wouldn’t they? Well, not if we are constantly resizing our bets. Let’s look at the original formula and see what happens.

Yamashita:
B’ = B*[(1+f)^W]*[(1-f)^L]
B’ = $100*[1.01^1*0.99^1]
B’ = $99.99

We have lost a penny somewhere. That lost penny comes from resizing our second bet. If we win the first bet then we will have a bankroll of $101. Our second bet will be $1.01, which brings our bankroll down to $99.99 when we lose it. Conversely, if we lose out first bet then we have a bankroll of $99 and we would bet $0.99 on our second bet. Winning that will bring our total to $99.99.

Both outcomes are equally weighted because we are dealing with actual wins and losses instead of theoretical win/loss rates. Your concern was with the fact that we should not be weighting the outcomes equally when we know that the theoretical win/loss rates are not the same. That is a very good point that is exemplified very well above. If we are betting optimally, why would we lose money in a fair game with equal numbers of wins and losses? We won the same number of hands that we lost, yet our bankroll is shrinking. How can that be?

The answer to that comes from the assumption that a Kelly bettor is only betting when he has the advantage. If the game were skewed in favor of the house and the player had that information a priori, he would not make any bets. In the formulas above we used a constant value of 1% for f, but in reality that would be adjusted for each bet. If the advantage changes, the bets would change. If the odds change, the bets would change. The formulas above assume a constant 1% advantage, which is not applicable to many casino games, but it is correct for that situation.

[EDIT: After a good night's sleep
I understand what you’re getting at. If we want to simulate BJ more accurately we could use a win rate of 43%, a loss rate of 49% and adjust v to the average payout (including splits, doubles, insurance and everything else AutoMonkey mentions below). That would weight the possibilities more accurately and give us a better idea of what our growth rate will actually look like. Substituting W=43, L=49, N=100 and v=1.175 (about a 1.5% advantage) gives very different results between those two formulas.]

But now it’s late and the Dodgers lost again. I’m going to bed. Perhaps I’ll be more coherent in the morning…or perhaps I’ll just erase this entire post. :sleep:

-Sonny-

 

[bj bob]

You can sleep well Sonny. Manny's wearing Dodger Blue tonight.

 

[Automatic Monkey]

In BJ we have quite a few possibilities other than a win loss or push. We have doubling or splitting, doubling and splitting, surrenders, bonuses for naturals, and lost insurance bets, and sundry combinations thereof.

What I think you might be looking for is that variance number that is used in Kelly bet sizing, and in BJ for a single hand that number is always somewhere around 1.3. I believe that number is a factor that takes all hand outcome possibilities into account. The fact that when we're counting we usually get an advantage around 1.3% gives us the rule of thumb that a full-Kelly bankroll is around 1000 units, assuming Kelly-proportional betting.

 

[zengrifter]

The fact that when we're counting we usually get an advantage around 1.3% gives us the rule of thumb that a full-Kelly bankroll is around 1000 units, assuming Kelly-proportional betting.

I would have said half that many units, but no math to suport my guesstimate, just spitballin ad David letterman would say. zg

 

[callipygian]

That lost penny comes from resizing our second bet. If we win the first bet then we will have a bankroll of $101.

I see my error now. That's exactly it - I'm sizing the bet at the initial bankroll and assuming f*B remains constant, which it doesn't. Thanks!

If we want to simulate BJ more accurately we could use a win rate of 43%, a loss rate of 49% and adjust v to the average payout (including splits, doubles, insurance and everything else AutoMonkey mentions below). That would weight the possibilities more accurately and give us a better idea of what our growth rate will actually look like.

That was exactly my next step, to re-calculate the Kelly criterion assuming there's going to be an informed decision on splitting and doubling (in which case the optimal bet might be smaller than otherwise assumed because doubling the bet would cause overbetting otherwise).

That's why I'm very interested in getting the nitty-gritty of the derivation down. :grin:

Thanks, too, Automatic Monkey.