This is a very interesting question that I think that deserves some analysis before answer it. Let's start saying that the Product Owner is usually appointed by the organization and not by the team.
Even though Scrum literature does say that the Scrum Team it's self governed and can appoint their own Scrum Master, it doesn't say much about who appoints the Product Owner and how could decide to remove him if necessary. My recommended rule of thumb would be this: the Product Owner-originally appointed by his organization-could be removed if the team agrees and present a business case that justifies this decision. Upper management endorsement would be required though.
In my experience, Product Owners have a great deal of talent and experience dealing with clients, they're part of the sales force; more clearly, they're the advance team with clients. They have a great ability convincing people-both clients and in house-and are in essence business driven individuals.
So, as you can guess, these types of individuals are not abundant. Besides, Product Owners require a deep technical knowledge, not only good selling skills.
I guess that the question have some triggers, why a team would be thinking in changing its Product Owner? Is it because o a lack or communication? Is it because the Scrum Master is not full filling his role? Before even in thinking in changing somebody my advice would be to drill down the root cause of the problem.