The basics of this is, Blood Boil scales with AP (and therefore Vengeance) and Heart Strike is not. Plus Blood Boil does equal damage to all targets while Heart Strike does 50% less for each target. Therefore, there are magic points where one is better than the other. Here are the numbers.
Targets BB AP Required
Essentially what it comes down to is how many targets are going to hit, how much AP do you have and what is the average damage on your weapon.
To see a more in depth spreadsheet of the basic, take a look at this spreadsheet: