Set up the regular Nextcloud install, instead of the AIO. The AIO is not required for any of the things you need, so you end up with lots of stuff you are not gonna touch.
I use Nextcloud with:
Calendar
Contacts
Cookbook
Notes
Tasks
Polls
Tasks can work together with the Tasks.org Android client and there are seperate apps for Notes and Cookbook, while Calendar and Contacts can integrate with your mobile apps via DavX5 using CalDav.