That depends entirely on your definition of freedom.
If someone were to punch me, it would violate my freedom from being punched/ bodily harm.
This is illegal in the US.
If someone were to insult me, it would violate my freedom from being insulted (maybe psychological harm).
This is legal in the US but illegal elsewhere.
speech alone is not enough to equate to action
Speech is not something that happens to you, it's something you actively do. It is always an action.