Use a weighted average: Let x = amount of 5% solution and y = amount of 20% solution
then (5x + 20y)/(x + y) = 8 and x + y = 3
So,
5x + 20y = 24
and x + y = 3
Solving the two equations we get x = 12/5 liters of the 5% solution
and y = 3/5 liters of the 20% solution