With Fantasy work, In my opinion get yourself pleanty of models of your houses / castles/ villages, variation of trees and plants etc first. I find that once the assests are built and you have grand variation of tress, plants, bushes etc it all comes together and you can use your props to get an average estimate of what height is what in your world that way your mountains etc would be at the proportions you want relative to your props. (This stands for anything open world (I am using Elder Scrolls style as my example))
The above is probably what I think is best as your estimated proportions would be much better compared to starting of with the terrain and it becoming much smaller or larger than your character.
When it comes to singular maps with the terrain tool in Unity I find it is a quick solution but does not often give the best results. I find building maps/scenes in a 3D Program and using them in your scenes with a mesh collider is the best option as everything is proportional to one another and you can scale it to the size of your character or scale the character to the size of your scene. An example of this is the FPS Tutorial from Unity. Though this method is only Ideal when your working in maps and not a open world
This is a design question which is very much forum based and not Unity Answers which is mainly directed at programming issues, though I gave you my opinion on the matter.
Hope this helps somehow. Plan it out, get used to the feel of the terrain tool.