Hi Luke!
Ropes and strings around pulleys are a surefire way to make a force/motion problem seem more involved than it actually is. Though we have one block moving horizontally, and another vertically, and we have the pulley in between, we can still basically deal with this as a one-dimensional motion problem. This is because, as long as the string is taut, there are only two ways the system of the two blocks can move: [Block B falls/Black A moves toward pulley] and [Block B rises/Block A moves away from pulley]. This is just like moving along, say, the x-axis, either left or right.
We can refer to the [B down/A toward] direction as "positive," and the [B up/A away] direction as "negative."
In this reckoning, let's set up Newton's 2nd law along this "axis" of motion. First, we define the system. I choose to wrap my "system bubble" around both masses, including the string between them. The two masses and the string (massless, with tension) are thus my system.
Then, let's identify the forces acting on this system in our defined "positive" and "negative" directions. The force pulling the system in the positive direction, you may be able to see, is the force of gravity on the hanging mass. This pulls B down and A toward the pulley; hence, positive.
In this case, there are no other forces acting along our axis of motion (no other pulls B down and A towards, and no force pulls A away and B up). So the force of gravity on B is the net force, along the axis of motion, for this system. [NOTE: The tension in the string between the blocks is a force INSIDE our system -- an internal force -- and so is not included in the force equation for this system; only external forces, from outside the system, matter]
Now, we can set up Newton's 2nd law along the axis of motion:
Fnet = FgonB = msysa
FgonB = mBg
msys = (mA+mB) (since our system is both blocks together.
So the Newton's 2nd law expression becomes:
mBg = (mA+mB)a
This yields an expression for the acceleration a, and also a way to evaluate Part (b).
Try this out; I hope it helps! Just let me know if anything is unclear or you have more questions about this.