Paul, There could be problems with the water content, too, which is why I hope to hear some more comments. There may be issues similar to concrete.
Regarding the brittleness, if it would work as a refractory material it would definitely be for one-time use.