I tried to contrast how my team’s parents acted and how the daughter’s father acted.
Your goal of building her self-esteem should be the goal of every parent. Sometimes parents forget it should be enjoyable for the child also. If it is, they will put everything they have as they participate or compete. Competition is good. If you mean a time that competing becomes destructive to self-esteem, I agree. You have to decide if it is her performance that is affecting her esteem or if it an outside force such as a parent or coach. You can support her if it a matter of her ability. If it is another parent or coach, find another place.
You have many years, it sounds like, in the enjoyment (and angst) watching her in whatever endeavor she desires and you support.
My wife and I followed my son from t-ball at 4, through soccer, lacrosse, basketball, college lacrosse, and some club lacrosse after college. My daughter played high school lacrosse, was a high school thesbian, college crew (rowing) and club rugby!
Now we follow as conflicting schedules permit: granddaughter and grandson – soccer, grandson -lacrosse, soccer or football, granddaughter -field hockey, and other challenges such as dancing. The joy continues!

But she’s not even in kindergarten yet.

I want her to find a physical outlet she enjoys and that she can continue to do for the health and fitness benefits long after high-school and college. I worry, though. I want these experiences to build up her self-esteem, not tear it down. DH and have already discussed the moment dance or gymnastics gets competitive, it may be time to pull her.