Hi Luke!
Splitting the chain into a "hanging" and "on the table" section will help, I think. In the ideal consideration, this falls into the same category as the question you posted about the two masses, one on a table and one hanging, connected by a string slung over a pulley. So I will appeal to that question in the setup; refer to that one for the concept I am using here.
The force that would pull the chain in the "down to the floor/toward the edge of the table" direction (which I will call positive) is the force of gravity on the hanging part of the chain.
The force acting against that, and thus pointing in my negative direction, is the force of static friction (assuming the chain to be stationary).
So, Newton's 2nd law along the axis of motion looks like:
Fnet = mHg - Fs-table = ma = 0 (for the chain being stationary)
where
mH = mass of the chain that is hanging off the table
Fs-table = force of static friction between chain and table
To have as much of the chain hanging off the table as possible, we assume we have the maximum static friction force trying to hold it on the table. As long as friction on this length of chain is acting simply -- which we always assume in this case -- we have:
mHg - μsN = 0
where
N = normal force of the table on the section of chain on the tabletop
μs = coefficient of static friction
The normal force of the table on the section of chain on top of it equals the weight of that section of chain:
N = mTg
where
mT = mass of the chain on the table
Thus, we have for our Newton's 2nd law expression:
mHg - μsmTg = 0
Now, with a constant mass per unit length (let's call that λ), we can recast the masses above in terms of lengths of the chain they represent.
m = λL (meaning a given length of chain L has mass m)
Thus:
mH = λLH
mT = λLT
And our Newton's 2nd law expression becomes:
λLHg - μsλLTg = 0
The λ and g cancel out, which leaves:
LH - μsLT
Thus, LH/LT = μs (meaning more length has to be on the table than hanging, which we might expect)
The fraction of the total length (which is just LT + LH) hanging off the table can be written as:
H = LH/(LT+LH) = μsLT/(LT+μsLT) = μs/(1+μs)
This meets the "minimum success criterion," in that it is less than 1, which a fraction of the total length should be.
I hope this helps! If you have any questions about it, or see any problems, just let me know.