Many states have multiple universities named after them, so you can have university of Pennsylvania and Pennsylvania state university.
Ohio state used to be " the ohio mechanical and agricultural school" when they were founded. They changed their name to "the Ohio state university" when they became a university. They were competing with "Ohio university" to be the main big state school, so the tried to emphasize it with "the" prounced with the long e (thee). It's become a bit of a meme since then. No one really cares but it's a funny thing for them to embrace and everyone else to make fun of.