Fare, I do not understand how you think about versioning. Why do you say semantic versioning is for binary libraries?
I even suppose you mean something different than me, Robert and others. Because semantic versioning as described at http://semver.org/ focuses on distinction between API compatible changes in libraries, and changes which break API.
