Yes, but only if you are reasonably sure that your credit cards have been stolen or are being used without your permission. In that case, call the credit card companies. They will close the account and reissue another card with a completely separate number.
Be sure to ask them to re-report the new account with the original account’s same “date opened” so that it doesn’t appear that you have opened a brand new account. The knee-jerk reaction to simply close every single one of your credit card accounts because you think someone may have had access to them is not a good idea, because in this way you will lose your credit history (especially bad if you have positive credit history) and your existing credit limits.