I have 100 Jira users but only 10 agents in Jira Service Management. Do I have to pay for the full 100?

Yep. This is consistent across all apps in the marketplace as set by Atlassian. The license size must match that of the underlying Jira instance.

In Atlassian’s words:

“You must purchase the app license that matches the highest Jira application tier. For example, if you have a 100-User Jira Software license, and a 10-Agent Jira Service Management license, your Jira apps must be at the 100-User level.”