The amortization chart shows that the proportion of your payment that is credited to the principal of your loan increases each year, while the proportion credited to the interest decreases each year. In the later years of your mortgage, more of your payment applies to principal and helps build equity. By refinancing late in your mortgage, you will restart the amortisation process, and most of your monthly payment will be credited to paying interest again and not to building equity.
A prepayment penalty is a fee that lenders might charge if you pay off your mortgage loan early, including for refinancing. If you are refinancing with the same lender, ask whether the prepayment penalty can be waived. You should carefully consider the costs of any prepayment penalty against the savings you expect to gain from refinancing. Paying a prepayment penalty will increase the time it will take to break even, when you account for the costs of the refinance and the monthly savings you expect to gain.
The monthly savings gained from lower monthly payments may not exceed the costs of refinancing. a break-even calculation will help you determine whether it is worthwhile to refinance, if you are planning to move in the near future.