You can play without fighting, but that is not really a good idea. The game is designed with fighting in mind, so generally speaking you´ll have an easier time if you do so.
Of course that depends on a lot of factors, especially where in the game you are. early on fighting is unnescessary, catering/negotiating is the more efficient option, but the more chapters you play the less efficient catering becomes and the more advantageous is fighting.
Squad sizes change around all the time depending on what you do, nothing to worry about there, it is what it is.
But if you have the option to decline a quest then decline away, look at the rewards, if they are not worth the effort hop over it, you have absolutely no negative consequences for doing so other than missing out on the quest rewards.
If you need help with fighting in general, look around here on the forum, there is tons of info on the topic or ask (preferably specific) questions. if you need help with a specific fight, feel free to post a screenshot 8and as detailed an info as you can give on what you have to work with, people will always be happy to make suggestions...