That's really an oversimplification. In the early history of the USA, there were two camps, the federalists and the anti-federalists. The federalists explicitly wanted a strong central government. The anti-federalists, which you might describe as the "small government" side, wanted a freedom from monopolies written into the constitution, with the understanding that the government would revoke corporate charters if any company got too powerful.

Even so, the federalists didn't really want an environment where strong corporations check the power of government, they wanted an explicit oligarchy where the most wealthy men around were Senators.

Neither of these camps really sound like modern libertarians to me.