This requires some analysis first to see what inequalities on a,b,c correspond to the geometrical conditions. For simplicity let's suppose a >= b >= c >= 0 (the other 5 possible orders of a,b,c are similar). Then a,b,c form a triangle if they satisfy the triangle inequality: a <= b + c By the law of cosines, an acute triangle would mean a^2 < b^2 + c^2.
> P[triangle]:= 6*int(int(int(1, a = b .. min(1,b+c)), b = c .. 1), c = 0 .. 1);
Maple needs a bit of help with this. It would have worked if we used piecewise instead of min:
> P[acute]:= 6*int(int(int(1, a = b .. piecewise(b < sqrt(1-c^2),sqrt(b^2+c^2),1)), b = c .. 1), c = 0 .. 1);
I don't know if Maple can do this complicated-looking integral symbolically. Here's the numerical value