Sounds Like you need to adjust your tiller. Start by making sure your limb bolts are adjusted evenly. I will attach an explanation of tiller. I didn't write this up but it is a good explanation of what you are seeing and how to fix it. Or Google "adjust Bow tiller".
If the grip, arrow rest and...