House Tax bleg

In July 1798, Congress enacted a direct tax to raise revenue for national defense against France. The “House Tax” imposed taxes on land, houses, and slaves. As required by Article I, section 9, clause 4 of the Constitution, this direct tax was apportioned by state population. Fries’s Rebellion, which was eventually suppressed by President Adams, involved violent resistance to this tax, based on the claim that the tax was unconstitutional. Because the direct tax was properly apportioned, it seems perfectly constitutional to me. Does anyone know the specifics of the constitutional objection to the House Tax?